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Executive Summary 



This report summarizes a series of tasks undertaken to assess to the feasibility of a longitudinal 
analysis of elementary and secondary schools. The initial objective of this Education Statistics 
Services Institute (ESSI) project was to see if it was technically possible to analyze the overlap 
samples of schools from the Schools and Staffing Surveys (SASS) to produce substantive 
findings. The overlap sample was initially used as a variance reduction device and was never 
examined substantively. 

Over the course of the project various other objectives were added and correspondingly a number 
of products have resulted. These additional products include an appraisal of longitudinal designs 
for a general study of schools; recommendations about the use of SASS for collecting 
information on the implementation and impact of recent school reform proposals; and 
recommendations about adapting SASS through various longitudinal designs. 

The body of this report provides two things. The first is a discussion and set of recommendations 
about the use of longitudinal designs for National Center for Education Statistics (NCES) data 
collections and reporting on schools. The second is a description of the initial feasibility study of 
the use of the SASS overlap study. The appendix includes other deliverables produced over the 
course of the project. 

In terms of the general proposal for a longitudinal study of schools, it is recommended that 
NCES not invest in a more elaborate panel design for the study of schools. In part, this 
recommendation is based on an assessment of the costs of designing useful items for such an 
undertaking, as well as other methodological burdens. In addition, NCES rarely uses more 
complicated results about organizational change even when they are available. However, the 
authors believe that NCES can derive a wealth of time-related information from its current 
surveys. In short, it seems unlikely that the payoff of a large panel study will be great enough and 
that the additional information would be fully used by NCES. 

In terms of the technical use of the SASS overlap sample, the study finds that it is feasible to use 
these data to enhance the repeated cross-sectional SASS with additional time-related information. 
At the same time, however, the analysis shows that many of the items that SASS currently 
collects about schools do not change much in the population over time. Some areas do show 
more change and these are the subject of a more detailed substantive analysis, which serves as an 
illustration of what NCES might be able to provide users from such an overlap sample. 

Since the overlap sample is a potentially valuable feature of SASS with few, if any, additional 
associated costs, it is recommended that future SASS administrations continue to include an 
overlap sample. 
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The Schools and Staffing Surveys (SASS) are a series of surveys that were administered to school 
district staff (“District Questionnaire”), school administrators (“School Administrator Questionnaire”) 
school administrative staff (“School Questionnaire”), and teachers (“Teacher Questionnaire”) during the 
1987-88, 1990-91, and 1993-94 school years. These surveys will be administered again during the 
1999-2000 school year. In each of the previous SASS administrations, approximately 9,500 public 
schools were surveyed — approximately 1 1 percent of the nation’s public schools. Schools were sampled 
for participation through stratified random sampling with probabilities proportional to size 1 . 

The degree of overlap for the public school sample was controlled and set at 30 percent (for the 
1990-91 and the 1993-94 school and administrator Questionnaires). This was done to decrease the 
likelihood of schools or districts not wanting to participate in subsequent waves and to reduce sampling 
error. The overlap sample for 1990-91 and 1993-94 public schools included about 2,900 schools. 

The sampled schools were used to select school districts. That is, if a school were sampled, the district 
in which it was located was selected to receive the district (Teacher Demand and Supply) 

Questionnaire. By controlling the degree of overlap at 30 percent, the degree of overlap of districts was 
estimated to be controlled at 58 percent. Even though many of the same schools and districts were 
surveyed in two or more waves of the SASS, the SASS was not designed with the purpose of 
longitudinal analyses in mind. The feasibility of using SASS Surveys, particularly the public school- 
level data reported in the Public School and Public School Administrator Questionnaires, was assessed 
in this project. 

SASS has used a newly drawn sample of districts and schools for each of its three data collections 
waves (1987-88, 1990-91, and 1993-94). Although successive surveys have added new items, many 
of the same factors were asked about in each wave. Therefore, unlike a one-time survey, SASS is a 
repeated, cross-sectional survey that can provide national and state estimates of variables at different 
points in time. However, SASS is limited in the information it can provide on the dynamics of 
educational change at the school and district levels. This information is only available for a subset of 
schools in a special sample because different schools are surveyed in the full SASS public school 
sample. 

This working paper has two parts. The first part looks at the costs and benefits of a panel study of 
schools for the overall NCES statistical portfolio. The second part is a demonstration of analyses of an 
existing partial panel of public schools that are followed in the course of the SASS administrations. 
Taken together these two parts outline the feasibility of a panel design for data collection of elementary, 
middle, and secondary schools. In the appendix are several other products from this project, including a 
summary memo report on the costs and benefits of the use of a longitudinal design to obtain 
information on schools and educational reform in the next SASS survey. 



1 Within each of the sampling strata, the probability of selection was proportional to the square root of the number of teachers reported on the 
Common Core of Data School Survey data file. 
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I. 



Should NCES Undertake Longitudinal Studies of Schools? 



A longitudinal or panel study refers to the collection of data from the same responding unit (district, 
school, or individual) at two or more different points in time. A cross-sectional study refers to the 
collection of data from different responding units at two or more points in time. Both types of study can 
allow statements to be made about how responding units have changed over time. 

There are benefits and costs associated with each of these types of study. In order to inform decisions 
about NCES’ undertaking a longitudinal study of schools, three specific questions 
are discussed: 

•What benefits would a panel study of schools add to the statistical portfolio of NCES? 

•What is the probability that these benefits will be fully realized within NCES reports? 

•What are the costs relative to the benefits of such a study? 



A. What Benefits Would a Panel Study of Schools Add to the Statistical 
Portfolio of NCES? 

Theoretical Benefits versus Practical Use 

On a purely theoretical level, it is hard to argue against the use of longitudinal designs to collect and 
report information on schools in the nation. Much of the public discussion about schools and their 
missions concerns change, either in terms of discussions about what to change and what to leave the 
same, or about what people suspect has changed about the operation of schools. The public assumes 
that educational change is a primary component of the education system. If the public image of 
education is surrounded in the language of change, shouldn’t the way one collects and report statistics 
about education directly incorporate change into study designs? This has been the logic of the existing 
longitudinal studies mounted by NCES. The panel surveys of students, such as High School and 
Beyond (HS&B) and the National Education Longitudinal Study (NELS:88), are designed around the 
notion that following a student’s educational trajectory over the school career is an approach that 
informs a variety of educational issues. 

Similarly, the various social sciences that ground our understanding of educational processes are often 
characterized as ventures in the study of change — such as individual change in learning, as change in 
job performance of a teacher, or as a change in the organizational environment of a school. The notion 
that the investigation of these behavioral and social phenomena requires longitudinal data is a common 
assumption. It is argued that these phenomena are dynamic — namely, that they involve change over 
time in quantities, events, duration, or transitions to new status — and therefore, only data collection over 
time will inform us of their true nature (e.g., Fienberg 1989). 

In the past several decades, considerable improvements in the survey techniques and statistical analysis 
procedures involving the use of longitudinal data (e.g., Kasprzyk, Ducan, Kalton, and Singh 1989) have 
been made. Starting in the mid-1980s various foreign governments increased their use of panel surveys 
to gather information about a range of social issues. In the United States, organizations such as the 
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Social Science Research Council and the American Statistical Association began to both develop and 
promote the use of panel designs for all types of surveys. This has lead to substantial capability in 
engineering such surveys and in analyzing the resulting data (Duncan and Kalton 1987). 

Given all of this, is it not simple to the answer the lead question in the affirmative? Indeed, others have 
made the same points, arguing for an NCES longitudinal study of teachers’ careers (Singer and Willett 
1 996). In the abstract, longitudinal designs for understanding schools are hard to reject. However, upon 
further reflection, the issue is far more complex, and cannot just be answered from purely a theoretical 
perspective. The answer is more a matter of practical use than technical capability. As a statistical 
agency with fixed resources, NCES needs to select wisely which innovations it should pursue and 
which it should leave to the research community. When the practicality of the panel data on schools is 
examined in relation to the kinds of information that NCES routinely publishes, it is not clear that 
longitudinal designs are the best way for NCES to collect information about classroom, schools, or 
districts. 

Figure 1 shows the kinds of information that can be derived from different survey designs. This table is 
taken from Bailar's (1989) general discussion of longitudinal designs and can easily be extended to 
specific educational surveys. The columns represent designs that increase by their degree of 
longitudinal sophistication. 

At the far left is the non-longitudinal single-time survey. An example of this type of survey from NCES 
is the typical Fast Response Survey System (FRSS) effort. Next, in terms of longitudinal sophistication 
is a repeated, no-overlap survey, often called a periodic survey. This is a recurring cross-sectional 
survey on the same topic. It asks similar or identical questions but does not administer different waves 
of the survey to the same respondents. Except for the technical, partial overlap sample, the SASS is an 
example of this design. 

A repeated, partial-overlap survey is a recurring survey on the same topic that rotates units. That is, a 
unit may be included in the first administration and then in a subsequent wave, but not necessarily 
included in all administrations. This design is usually employed to reduce variance in the estimates. 
Within the full SASS, the partial overlap sample is a good example of this design. 

A longitudinal, no-rotation survey (referred to hereafter as a panel study) is the most basic panel or 
longitudinal design. A panel of units is measured at each wave of the survey to create a full longitudinal 
record of information. NELS:88 is a prime example of this kind of design. 

Longitudinal rotation surveys are panels that add units at specified time based on substantive issues in 
what may be happening with the population over time. In other words, adding new units is not just a 
technical “freshening” of the sample, but is done to capture rare events in the population or compare 
cumulated effects of new versus old members within the population. 
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Figure 1. Longitudinal survey design by kinds of information* 
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♦Adopted from Bailar (1989), p. 3. 

The rows in this figure display the different types of information that are obtainable from each design. 
For example, the single-time design can yield one point in time estimates, estimates of durations, 
transitions, and the frequency of occurrence of events, and also inform about the relationships among 
characteristics. In terms of information about education, these could be estimates of the percentage of 
schools using site-based management (point in time), the length of time this approach has been in place 
assuming such a retrospective item is asked (duration), and an estimate of whether schools of certain 
sizes are more likely to use this management approach (relationships among characteristics). 

The repeated, no-overlap design yields these three kinds of information plus net change, trends, and 
rare events. An example of net change is the net change from the first to the second survey 
administration in the national mean percent of schools using site-based management. With at least three 
administrations, a trend (e.g., means at each time point) in this percentage could be described. 
Furthermore, one could show that perhaps because of training that took place between the first and 
second administration of the survey, new school principals who had never heard of site-based 
management were almost non-existent in comparison with their prevalence during earlier 
administrations of the survey. 
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A repeated, partial-overlap design yields all of these types of information, plus gross change (also 
called flow or churning). Since a partial overlap provides a partial panel, gross change can be derived to 
some degree. An example of this would be the percentage (not just two mean percentages) of schools 
that changed from non-site-based management at the time of the first wave to site-based at the time of 
the second wave. 

The two full longitudinal (panel) designs yield all of these types of information, with the exception that 
the longitudinal design with no rotation does not allow for certain kinds of rare events to be included. 
For example, if some unique event occurred between the first and second wave of the survey which 
resulted in the formation of a small number of a new type of school, and if these new schools adopted a 
distinctly different approach to site-based management, estimates of the impact of that event would be 
missing from a non-rotating panel design. Otherwise, both these designs yield information to estimate 
change over longer periods (assuming the panel is extended into time) and to estimate relationships 
between characteristics and change. As an example, the relationship between school size and the 
likelihood that a school changes to site-base management can be determined. Even the very complex 
information on what kind of change in size is related to what kind of change in management approach 
could be estimated. 

In sum, figure 1 shows that an impressive amount of time-related information can be derived from 
simpler designs. Even though the more longitudinally sophisticated designs add some unique 
information on change, the simpler, non-longitudinal designs still enable the description of time related 
phenomena. 

B. What is the Probability that these Benefits Will Be Fully Realized within 
NCES Reports ? 

NCES rarely reports complex longitudinal information, even when the capability exists. Most NCES 
reports primarily include the first three types of estimates described above (one point, retrospective 
occurrences, and relationships). The fourth type of estimate (net change) is sometimes reported; the fifth 
type (trends), rarely reported. Even the panel surveys that NCES currently conducts rarely, if ever, 
generate the most sophisticated longitudinal analyses possible. In part, this is because more 
complicated longitudinal analyses do not easily fit within the usual NCES report. In addition, given that 
policymakers typically use only the most basic information, the full capabilities of a panel design 
currently are not (and probably would continue to not be) used. Even though policy questions are often 
set in terms of change over time, the real questions underneath them are simple time questions. For 
example, the question, “Are more schools now using site-based management?,” is implicitly about 
change, but it does not require a very sophisticated design to provide at least a basic answer. The 
additional value of more sophisticated measures of change is not obvious to either the lay public or to 
most policymakers. 

Therefore, on a practical level, NCES does not often use the kinds of unique information that panel 
designs yield. The agency’s time-related information needs can be met with an array of useful 
information from simpler designs, and even when it has the capability for more sophisticated analyses, 
it rarely decides to carry these out. As is shown in the description of designs above, simpler surveys can 
yield a wealth of results that have some element of a time component. From a practical perspective, a 
fully longitudinal panel of schools would probably be used in the same way as the current, simpler 
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designs. The prestige in the statistical world associated with the use of panel designs may outweigh 
their actual contribution to the agency’s statistical portfolio. 

Nonetheless, one could argue that the case of school reform presents clear policy questions that NCES 
could approach with more sophisticated panel survey designs. For example, if a school adopts particular 
reform strategies (one kind of change), is that related to positive outcomes in students (another kind of 
change)? Of course, the best research design to answer this question is experimental, or at least quasi- 
experimental. National surveys are not the best methods for examining this kind of a question. 

However, since policymakers often wonder what will happen as reforms are adopted on a national scale, 
a survey with some longitudinal component might be considered. Therefore, it is useful to think about 
the main question of a panel study of schools regarding the specific issue of reform and NCES’ 
capability to provide information on this topic, which is discussed in more detail below. 

School Reform and Longitudinal Designs 

Regardless of whether one or many reforms are of interest, basic questions about reform that 
longitudinal information about districts or schools could help to answer include: 

•Where, how, and when are these reforms being implemented? 

•How are these patterns of implementation changing over time? 

•What school outcomes, if any, are attributable to reforms? 

In the case of the first question, longitudinal information could provide more and better information 
than is obtainable through a cross-sectional survey. As previously described, simpler longitudinal 
designs would provide a significant amount of information relative to the first two questions. For 
example, the repeated cross-sectional SASS could provide information on the numbers of schools that 
use site-based management procedures at each administration of the survey, provided the items are 
included each time. A panel design would enhance that useful information with more detailed 
information on which schools changed to more (or less) site-based management over a specific time 
period. 

The third question requires a panel design for a valid answer, provided that other more controlled 
experiments are not undertaken. Without pre-reform measures of school outcomes like achievement, 
teacher satisfaction, drop-out rates, and so forth, attributing effects to reformed practices becomes 
questionable. 

Without a doubt, the use of a longitudinal design to study reform seems attractive on the surface. Even 
following a panel across just two time points has clear appeal and utility for assessing the impact of new 
practices. Reforms are really changes in operating policy and practice, and thus they can be thought of 
as quasi-experimental treatments whose impact can be assessed by comparing a school’s functioning 
before and after implementation. Of course, there are also more elaborate panel designs with rotation 
available to compare reformed schools with unreformed schools over the same time periods. 

In addition to assessing the impact of reform efforts, panel designs may be used to gather information 
on factors associated with the adoption of reforms and how reforms are implemented across districts 
and schools. The political environments of districts and schools are complex and the impetus for reform 
comes from a variety of sources with little, if any, centralized policy-setting in the nation. 
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Therefore, although the national reform message may be more or less clear, reform is locally adopted 
and implemented. Some districts and schools will adopt reform while others will not, and the timing of 
adoption will vary as well. Also, among those that choose to adopt reform, actual implementation can 
take on different forms. Thus, although national education politics often produce pressure for 
educational change, the results of this pressure, falling upon a localized system, can yield substantial 
variation in reformed practice from location to location over time. Examining implementation over time 
across a panel of districts or schools is an attractive way to gather useful information on how reforms 
spread across the nation. 

It is true that multiple-waved panel designs provide more and richer information than simple cross- 
sectional designs. But it is not clear that NCES would be willing to use these kinds of designs to their 
fullest capacity. There is already some information on school change available in SASS that has not 
been used. In addition, the use of a panel study for evaluating school reform and other topics can be 
quite costly. 

C. What are the Costs Relative to the Benefits of Such a Study? 

The widespread belief that panel studies are more expensive than cross-sectional studies is not well 
established. Even though certain large longitudinal studies (such as High School and Beyond and 
NLS72) were undeniably expensive, their costs were largely a result of the number and special 
characteristics of the individuals studied. Such longitudinal studies can also be expensive because of 
costs associated with tracking respondents. However, for the SASS School Administrator and School 
Questionnaires, respondent tracking costs would be negligible. Additionally, costs associated with 
selecting new samples and with eliciting the initial cooperation of respondents that characterize cross- 
sectional surveys are negligible for panel studies (Pearson, 1991). 

Operations Costs versus Design Costs 

The cost of conducting a panel study of schools will be discussed in the context of studying educational 
reform for illustrative purposes. Additionally, we will assume that SASS would be the basic host survey 
for such a panel design. Even though this discussion of costs is specific to SASS and reform, it applies 
to any other educational topic that could be addressed using a panel design. Since sample selection and 
respondent tracking costs would be minimal, the administrative costs of implementing a panel survey of 
districts and schools would most likely be no greater than the administrative costs associated with a 
repeated cross-sectional survey (with or without overlap). However, the development of a panel survey 
that fully maximizes the time dimension in its item structure would require considerable additional 
investments. In the SASS context, this would require extensive redesign. 

Taking full advantage of the panel survey design requires extensive item development efforts. Analyses 
of the SASS overlap sample (see Feasibility Study results below), using items which were simply 
repeated in subsequent SASS waves, over a three year span, indicated that there is not much basic 
structural change in schools (e.g., grade-span, school size, staff size). Probably most of the short-term 
change in schools has been in policies and practices. For example, NCES’s study of tracking policies 
found that almost sixty percent of schools had changed their tracking policy in the last five years and 
about one half of schools were considering new modifications to schools (Carey, Farris, and Carpenter, 



A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 



7 



1994). Changes in school operation, not structure, are how most reforms would be implemented, but 
operations of schools are hard to measure. Also they may change often, particularly in politically 
contested areas of schooling. Since the current SASS does not focus very much on the dynamics of 
school policy setting and operation, there would probably not be much added information from just 
repeating current SASS items to a full panel of schools. To make SASS items yield more information, 
particularly about policy and practices, extensive item re-design, especially for the proposed six-year 
spell between data collection waves would be necessary. 

For example, although it is not recommended here, one could propose to re-sample the entire 1993-94 
sample, adding retrospective items on policy and practice changes related to specific reform issues such 
as school-based management. Developing relatively easy to answer items that will elicit valid responses, 
and that are retrospective to six-years on school operation for all of the different types of schools 
included in the full SASS sample would be a large and costly undertaking. Furthermore, items which 
require schools to dip into their institutional memory are easily prone to measurement error. 
Accordingly, valid items would take extra design efforts. In addition, there are response burden costs 
hidden in an extensive retrospective panel study. Items that are more sensitive to time issues usually 
require more response time. Finally, since respondent burden must be maintained at reasonable levels, 
the addition of new items would most likely require elimination of items measuring other factors. 

NCES would also face numerous other technical costs in conducting a panel study of schools. Far more 
is known about measurement error in the responses of panels of individuals than is known about the 
responses of panels of organizations (Kasprzyk et al. 1989). This is further complicated when the units 
are organizations like schools which may have different individuals serving as the designated 
respondent for different waves. Analyses of the SASS overlap sample showed there were a significant 
number of schools with a different principal (the designated respondent for the School Administrator 
Questionnaire and a possible respondent for the School Questionnaire) from one wave to the next. 
Following a panel of organizations like schools and attempting to collect complex information on these 
organizations represents a formidable design challenge (Colledge 1989). 

Would the costs for significant longitudinal re-design be justified by the information payoff for NCES 
on the topic of reform? School organizational reform is politically important to a large part of the 
American educational establishment and the public. And as described in Pechman et al. (1996), many 
of the current reforms focus directly on the general areas of school management that SASS tried, in a 
limited way, to capture on the last two waves. As described above, longitudinal designs are useful for 
collecting information about the impact, the spread, and the variation in reforms in districts and schools. 

However, to justify the costs of a large, longitudinal re-design, NCES would have to be convinced that 
reports on the more complex nature of school change, as outlined above, are worth the investment. To 
maximize an investment in a longitudinal survey of schools on reform, NCES would also have to plan 
to invest in far more complex analyses and reports than it currently does. 



Summary 

The specific recommendations below about re-designing SASS for a panel study of reform apply to the 
general question of the value of a panel study of schools for NCES. On the whole, the specific 
recommendations listed below caution against such an undertaking. Given the large amount of time- 
related information already underutilized by the agency, it is difficult to argue that the added costs of a 
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panel survey would be used differently and add significantly to the agency’s reports on schools in the 
United States. (This is not to suggest that NCES should not make use of short term longitudinal designs 
with surveys, such as a pre-post test design in achievement testing.) 

Specific Recommendations about Longitudinal Study of School Reform 

Although a longitudinal design for SASS would provide useful information on educational reform and 
would also strengthen NCES reporting capabilities, the re-design costs and the impact associated with 
changing questionnaire items mitigate against a full panel, longitudinal study for the entire SASS. 
Instead, small-scale, longitudinal designs that are alternatives to a full panel design should be 
considered as a way to capture important time variant information about reform practices. 

Operational costs for a panel design of districts and schools are not significantly higher than those for a 
repeated cross-sectional design. However, costs to re-design SASS items to take advantage of the time 
dimension of a panel design can be substantial. These new items may also impose a greater burden on 
respondents. It does not seem advisable to design retrospective items for any sample of 1993-94 
schools. Instead, one should consider starting prospectively, either with a subsample of the 1993-94 as 
baseline, or with a subsample of the 1999-2000 as baseline. 

Organizational impact (outcomes) of reforms will be relatively hard to measure in SASS, regardless of 
the longitudinal nature of the design. But information on impetuses for reforms and implementations of 
policy and practices associated with many current reforms can be accurately collected. 

II. A Technical Feasibility Study of the Overlap Sample in SASS 

This section reports on two sets of analyses. The first set of analyses describe how schools have 
changed over time. These changes are cross-tabulated by school characteristics. The second set of 
analyses uses exogenous data to assess the relationships between state education policy activity and 
changes in the influence of various actors in school decision-making practices. This section includes a 
brief description of the steps undertaken to prepare for the analyses, a discussion of issues that must be 
considered in interpreting longitudinal findings, and an outline of the variables and procedures that 
were employed in these analyses. This is followed by the results of these analyses and indications of 
potential utility of longitudinal rather than cross sectional approaches. A set of tables, presenting a 
diverse array of findings are included. 

A. Preparation for the Analyses 

Determining which panel analyses should be conducted 

There are three possible overlap samples for panel analyses of SASS schools: 

Panel 1 1987-88 to 1990-91 
Panel 2 1990-91 to 1993-94 
Panel 3 1987-88 to 1993-94 



A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 



9 



Between 1987-88 and 1990-91, the SASS surveys underwent substantial revisions. As a result, most 
of the 1987-88 SASS survey items underwent minor or major changes in the 1990-91 SASS. For the 
SASS Public School Questionnaire, across all three waves, there were NO ITEMS relating to the 
following areas that were asked exactly the same way: 

Enrollment counts 
Minority enrollment 
Male enrollment 
Admission requirements 
Length of school day 
Type of school 
Number of students absent 
Number of staff (by categories) 

Minority teachers 
Teachers absent 

Programs and services available (and number of students participating) 

Library/media center presence 

Kindergarten program (presence, day length, week length) 

Chapter 1 participation (enrollment, staffing) 

National School Lunch Program participation (number eligible, approved, and participating) 
Vocational/technical program presence 
Grade 12 (presence, previous enrollment) 

Graduation rate 
College application rate 
Teacher vacancies 
Methods used to fill vacancies 
Difficulty in hiring teachers 

Most of the noncomparabilities are associated with 1987-88 items. Nonetheless, using stringent criteria 
(that is, definitions, item completion instructions, and complete question wordings must be identical), 
only two of the above items were identically worded in 1990-91 and 1993-94: 

College application rate 
Teacher vacancies 

However, the non-identical items administered in 1990-91 and 1993-94 were more similar than the 
1987-88 items were to the 1993-94 items. Similarly, the Public School Administrator Questionnaire, 
the other SASS Survey dealing with school related issues, asked about the following topic areas: 

Principal demographic characteristics 

- Degrees earned, majors, year received 

- Teaching experience (years, last teaching assignment) 

- Other school positions held 

- Aspiring administrator training 

- Training for current position 

- Experience as principal elsewhere 

- Other school administrative experience 
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- Non-education experience 
Planned tenure as principal 

Salary 

Length of employment contract 
Benefits received 

School environment (extent to which certain matters are a problem)Decision-making influence 
of people or groups, re: 

- Establishing curriculum 

- Hiring new full-time teachers 

- Setting disciplinary policy 

- Deciding how the school budget will be spent 

- Determining content of in-service programs 

- Evaluating teachers 
Importance of different educational goals 

As with the School Questionnaire, none of the above items were asked in identical ways across all three 
waves. And, as with the School Questionnaire, most of the noncomparabilities are associated with 
1987-88 items. Appendices A and B provide comparisons of the item wordings used to measure 
similar constructs across the SASS waves. 

Since the 1990-91 and 1993-94 SASS surveys were the most comparable and contained a larger set of 
items than the 1987-88 survey, analyses focused on panel 2 and consisted of about 2,900 public 
schools. Some attempts were made to analyze comparable items in panel 3. Panel 3, the six year sample, 
was smaller and consisted of about 950 public schools. Because of the longer time period, panel 3 was 
deemed worthy of some analytic effort. 

Obtaining Matching Codes and Constructing Data Analysis Files 

The data collector for SASS, the United States Bureau of the Census, provided a data file with IDs for 
linking 1990-91 school data to 1993-94 school data. Pinkerton Computer Consultants, Inc. provided a 
file with linking IDs for the 1987-88 and 1990-91 data. SASS Survey data were provided by the 
United States Department of Education. The SASS Survey data included adjusted case weights, to 
compensate for nonresponse and to insure that sample totals would correspond to sampling frame totals. 
These data were used to create analysis files for the SASS School Questionnaire and the SASS School 
Administrator Questionnaire. Variables were recoded to facilitate their use in longitudinal files. 2 
A large data item comparability study was undertaken to examine substantive similarities and 
differences across years. Results of this analysis are presented in the figures that appear after the tables 
below. 



2 Variables were renamed according to the following conventions: 

1987-88 1990-91 1993-94 

School data AS****** BS****»* CS****** 

Administrator data AA****»* BA****»* CA*****» 

The first character of all variables represents the year of the survey and the second character represents the questionnaire. For example 
CSTOTENR (total enrollment) is from 1993-94 school questionnaire, BASALARY (principal salary) is from 1990-91 administrator 
questionnaire. 
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Deciding on Analyses 

During several team meetings in which 1987-88, 1990-91, and 1993-94 Public School Questionnaire 
and Public School Administrator Questionnaires were reviewed to identify comparable survey items 
across these three survey year periods, it was decided to divide the analysis of organizational change 
and stability into four parts: 

1 . Gross structural change analyses 

2. Analyses of organizational leadership change 

3. Descriptive analyses of other school characteristics 

4. Analysis to identify patterns among schools in terms of organizational 
change and stability 

In the second study, associations between changes in the decision-making roles of various groups and 
individuals and state education agency policy activity in several domains (accountability policies and 
school-based management policies) was investigated. 

B. Issues which Must Be Considered When Interpreting Longitudinal Findings 

There are several technical issues related to using SASS data to conduct analyses over time. They 
illustrate potential problems in future analyses of partial panels of schools. In some cases the problems 
have obvious solutions, in other cases additional analyses were required. These issues include: 

Absence of new schools. By definition, no school less than 3 years old can be included in the panel 2 
analyses; no school less than 6 years old, in the panel 3 analyses. Changes reported are generalizable to 
only the set of schools that have been in existence more than 3 (or 6) years, and may not be 
characteristic of newer schools. 

Salary items. Salaries can be converted to constant (e.g., 1993-94) dollars using inflation factors from 
Consumer Price Index for the various years. 

Student data. Whenever possible, both absolute size change and percentages (i.e., rather than the 
number of Chapter 1 students served, the percentage of students served by Chapter 1 programs; rather 
than number of graduates, a graduation rate based on the number of 12 lh graders enrolled in the previous 
year) will be reported. 

Non-comparability of items: Item wording. As previously indicated, numerous SASS items were 
modified in efforts to improve data quality. Unfortunately, any change in item wording may have effects 
on the responses elicited. If an item was functioning as intended, there would be no need for 
modification. The fact that an item was modified strongly suggests that there was a belief that the item 
was not working (for all respondents). Accordingly, when a new version of an item is developed, there 
is an expectation that it will elicit different (and more valid) responses from some respondents. So, the 
fact that an item has been reworded indicates that there is a strong expectation that, at least for some 
respondents, it will elicit a different response than the previous version. 
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It has long been known that minor rewordings of attitudinal items can render them noncomparable. 

_ Many believe that this will not be true for the wording of factual items. However, the literature clearly 

shows that factual items are also sensitive to wording and formatting effects. For example, principals 
who completed the School Administrator’s Questionnaire (in both 1987-88 and 1990-91), were 
reinterviewed a few months after completion of the survey and readministered factual items about their 
education. In 1987-88, educational attainment was measured through the following item: 

£ Which of the following college degrees have you earned? (Mark all the degrees that you have 

earned.) 

a Associate degree or vocational certificate 
o Bachelor’s degree 
a 2nd Bachelor's degree 
a Master’s degree 
o 2nd Master’s degree 

a Professional diploma or education specialist (At least one year beyond M.A. level) 
o Doctorate (e.g., Ph.D., Ed.D.) 

D First Professional degree (e.g., M.D., L.L.B., J.D., D.D.S.) 

§ Upon readministration in 1987-88, the bachelor’s degree item had a gross error (discrepancy) rate of 

20.3 percent; the master’s degree item, a gross error rate of 9.9 percent! So, in 1990-91, the items were 
reformatted to ask: 

Do you have a bachelor’s degree? 
a Yes 

• a No 

Do you have a master’s degree? 
a Yes 
a No 

^ With this rewording, the gross error rates were reduced to 1 .3 percent and 1 .7 percent, respectively. 3 

Most item rewordings are not expected to have such substantial effects. 

Non-comparability of items: Context effects. Even when identical items are administered, it is not 
always possible to make similar comparisons over time. If there is a difference in the sequence in which 

fr some items were presented, item context effects may be manifest. An example of possible context 

effects is provided by the School Administrator Questionnaire item one asking about the extent to 
which various matters are a problem in the school. In the 1987-88 School Administrator Questionnaire, 
item 14 asks: 



3 NCES (1994). Quality Profile for SASS. Aspects of the Quality of Data in the Schools and Staffing Surveys (SASS). Washington, D.C.: 
Government Printing Office, NCES 94-340; Tushery, J , Royce, D., and Kasprzyk, D. (1992). The Schools and Staffing Survey. How 
Reinterview Measures Data Quality. Proceedings of the American Statistical Association, pp. 458-463. Alexandria, VA: American Statistical 
Association. 
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For each of the following matters, indicate whether it is a serious problem, a 
moderate problem, a minor problem, or not a problem in your school. 

Thirteen items followed. In subsequent years, most of the same items were listed. However, additional 
items were added and some were deleted. Since most of the items were asked in subsequent waves, 
simple longitudinal comparisons of responses seem possible. Unfortunately, attitudinal items are 
sensitive to context effects. That is, the previous items in the list create a context (or set) and can 
strongly influence subsequent responses. Specifically, in the 1987-88 School Administrator 
Questionnaire, “Physical abuse of teachers” preceded “Verbal abuse of teachers.” In subsequent waves, 
“Physical abuse of teachers” was eliminated. It can be argued “Physical abuse” creates a context that 
minimizes the seriousness of “Verbal abuse.” So, if there were a decline in the perceived level to which 
“Verbal abuse of teachers” was a problem, it could be attributed to either a real change in school 
climate or to context effects. The literature is very clear — preceding items can strongly influence 
responses to subsequent items. Unfortunately, the literature does not allow one to estimate the 
magnitude of these effects a priori or to predict with certainty that they will occur. 

Dealing with non-comparability of items. To address this issue, it was assumed that context effects 
are randomly distributed among principals. This enabled comparisons of the decline in the level to 
which this was a problem in elementary schools vs. secondary schools and in urban vs. rural schools. 
So, if there was a substantial decline in one type of school (say, urban schools) relative to other types of 
schools, one could assert that this decline represented a real change in climate. However, if analyses by 
school characteristics show comparable declines across all the different types of schools, one cannot 
know (with certainty) if the decline is a context effect or a real decline. 

Another approach one might take to address this problem is a normalization approach. That is, one can 
look at the average, overall change across all items and then compare the change in a specific item with 
the overall change. This approach assumes that the entire list provides a context for respondents and 
that items appearing before and after the item of interest will contribute to contextual effects. (Although 
this has been demonstrated to occur, particularly in a self- administered questionnaire when the 
respondent reads the entire list or changes responses after completing subsequent items the impact of 
the immediately preceding items will be probably be greater for most respondents.) 

Interpretation of responses to the school decision making influence items must take into account the 
existence of context effects. These items, in 1987-88, asked about the influence of “School 
district/Goveming Board,” “Principal/Head,” and “Teachers.” In 1990-91, the number of potential 
actors increased by three and “School district/Goveming Board” became “School Board.” In 1993-94, 
the list of actors included two new categories and “School district/Goveming Board” was broken up 
into two categories: “School district staff’ and “School Board.” These items are still valuable in 
identifying the types of schools in which change was the greatest. One has to be cautious about 
asserting that overall change has occurred for these items. 

Missing data. In the 1990-91 and 1993-94 SASS, item nonresponse was nonexistent, since missing 
data were imputed. In the 1987-88 SASS, there was no imputation. Accordingly, issues of response 
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bias must be considered when 1987-88 SASS questionnaire responses are compared with those of 
subsequent years. 4 



C. Variables that Were Employed in the Analyses 

Gross Structural Change Analyses 

Schools change their grade structure. These changes most often are small and involve the addition or 
subtraction of pre-kindergarten and kindergarten programs (Levine and McLaughlin, 1996). In order to 
investigate the how schools changed, their grade range of students served in 1990-91 and in 1993-94 
were compared. Changes and stability in enrollments, faculty, and student/teacher ratios were also 
examined, using survey variables providing measures of total student enrollment, number of teachers, 
and a derived student/teacher ratio. 

Analyses of Organizational Leadership Change 

The SASS School Administrator Questionnaire and the SASS School Questionnaire contained the 
items that were compared within the overlap sample panel of schools. For all waves of the SASS, the 
principal was the designated respondent for the School Administrator Questionnaire. However, schools 
might have had different principals for each administration of SASS. 

The respondent for the SASS School Questionnaire was determined by the school’s principal. That is, 
School Questionnaires were sent to principals, without any restrictions on who may complete them. 
Principals could complete the questionnaire themselves or they could assign someone on their staff to 
complete the questionnaire. Since the items on the School Questionnaire were all factual (rather than 
attitudinal), changes in respondent should have less of an effect on responses. 

In the 1990-91 Administrator Questionnaire, Item 9a asked: 

Prior to this school year, how many years have you been employed in each of the following 
positions? 

a. As the principal in this school? 

Similarly, in the 1993-94 Administrator Questionnaire, item 16 asked: 

Were you the principal of this school in the spring of 1991 ? 

These items were used to identify principals who were (or should have been) respondents to two or 
more of the survey waves. A few invariant characteristics of principals were investigated to verify 
whether the principal was really the same respondent in different waves: 



4 The new SASS CD-ROM, which was released after these analyses were conducted, contains imputed data for the SASS surveys. 
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• Year of birth 

• Gender 

• Race/ethnicity 

• Hispanic origin 

• Years of teaching experience prior to becoming a principal 

• Other positions held before becoming a principal 

• Previous educational experience 

These invariant characteristics were relatively stable among individuals responding to the 1993-94 
School Administrator Questionnaire who indicated they were the principal of this school in the spring 
of 1991. There were several exceptions. This might reflect the fact that in some schools, more than one 
person fills the role of principal in the case of co-principals. 5 And different co-principals could have 
responded to the survey in the different waves. 

To minimize the impact of different respondents in our analyses of change, attempts were made to 
restrict the panel schools to those that had the same respondent to the School Administrator 
Questionnaire. New leadership can bring change, but also a different informant on an organization can 
have different responses to items than his or her predecessor. While we cannot completely disentangle 
these effects, the analysis sheds some light on this issue in the longitudinal study of schools. 

Descriptive Analyses of Other School Characteristics 

After reviewing the High School and Beyond (HS&B) longitudinal results from the National 
Longitudinal Study of Schools (Levine, 1996), a set of variables from the SASS School Administrator 
and SASS School Questionnaires was selected. These variables were chosen for inclusion in the 
descriptive analyses, reflecting their potential to provide information about whether or not schools 
change. These variables included: 

(Question numbers are 1993-94 question numbers) 

Administrator Questionnaire 

A24. To what extent is each of the following matters a problem in this school? 

• Teacher absenteeism 

• Student tardiness 

• Student absenteeism 

• Students cutting classes 

• Students dropping out 

• Student apathy 

• Physical conflicts among students 

• Robbery or theft 

• Vandalism of school property 

• Student pregnancy 



’ Levine. R. ( 1996 ). Changes in America’s High Schools. 1980-1993: The National Longitudinal Study of Schools. Palo Alto, CA: 
American Institutes for Research. 
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• Student use of alcohol 

• Student drug abuse 

• Student possession of weapons 

• Student disrespect for teachers 

• Verbal abuse of teachers 

• Lack of academic challenge 

• Lack of parent involvement 

• Parental alcoholism and/or drug abuse 

• Poverty 

• Racial tension 

NOTE: This item is subject to context effects, as discussed previously. 

A25. Using the scale 0-5, where 0 is none and 5 is a great deal, indicate how much ACTUAL 
influence you think each group or person has on decisions concerning the following 
activities? (Only three activities are comparable; comparisons with 1987-88 are not 
recommended. This item is subject to the possibility of moderate context effects.) 

NOTE: This item is also subject to context effects that mitigate against comparisons with 1987-88 data. 
Furthermore, comparisons in subsequent waves are only possible in three areas of decision making. 

School Questionnaire 

58. What was the total number of students enrolled in this school around the first of 
October? 

59. Around the first of October, how many students were: (RACE/ETHNICITY)? 

S30c. How many students graduated from the 12 th grade last year? 

S30d. How many of last year's graduates applied to two- or four-year college year? 

Analyses to Identify Patterns among Schools in Terms of Organizational Change and Stability 

One of the most interesting things in looking at changes is identifying the types of schools in which the 
greatest or least change has occurred. Due to time and resource constraints, attempts were only made to 
identify first-order effects (i.e., changes in high vs. low minority schools) rather than higher order 
effects (changes in urban, high minority schools vs. changes in urban, low minority schools). The 
independent variables used for cross-tabulations of findings were: 

1 . Grade range of school: Elementary, Secondary, Combined 

2. Region: North, East, West, South 

3. Minority composition: dichotomized as less than 20% and 20% or greater 

4. School size: <150; 150-499, 500-749; 750 or greater 

5. Urbanicity: Metropolitan status code (urban area, primarily inside central city 
[“urban”]; urban area, primarily outside central city [“suburban”]; non-urban 
area [“rural”]) 
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Analyses to Investigate Associations Between Changes in the Decision-making Practices and 
State Education Policy Activity 

Items from the School Administrator Questionnaire dealing with school decision-making practices in 
the areas of discipline policy, curriculum, and new teacher hiring were investigated. These items looked 
at the influence of various groups (state department of education, school board, principal, teacher, and 
parent association) in making these decisions. 

Information about the policy environment in which these changes occurred was obtained through a 
survey of representatives of 50 state education agencies (SEAs) and the District of Columbia’s 
education agency (Levine and Huberman, 1995). Data from this survey provided information about six 
different domains of policy activity: 

1. Performance Reporting and Dissemination (Accountability) 

2. Testing Policies 

3. Curriculum and Instruction Policies 

4. Public School Choice and Enrollment Policies 

5. Teacher Policies 

6. School Organization Policies 

These domains were chosen because they were believed to be areas of significant policy activity, they 
could be related to student achievement, and they could have effects that could be measured by re- 
administering High School and Beyond survey items. Policy activity in all of these domains increased 
between 1980 and 1993. 

Performance Reporting and Dissemination (Accountability) 

Accountability policies prescribe the reporting and dissemination of information about school 
performance. Policies in this area usually do not make value judgments about school performance 
levels. That is, most states do not specify performance standards or tie rewards and sanctions to levels 
of performance. When rewards and sanctions are specified, they are most commonly in the form of 
good or bad publicity. 

Policies related to performance reporting can vary in many ways, including the type of information 
disseminated, the audiences to whom they are directed, the standards they specify (if any), and 
incentives and sanctions associated with these standards. Required reports can summarize performance 
for individual schools, districts, or the state as a whole, placing public pressure on the specific agency 
whose performance is being reported. The reports can also contain contextual information to help users 
interpret the information. 

Because of these factors, individual states can differ tremendously with respect to the intensity of their 
accountability policies. One state may widely disseminate school, district, and state-level measures of 
performance, specify standards that schools and districts should attain, provide rewards to high 
performing schools and agencies, and apply sanctions against poor performing schools and agencies; 
another state may simply report statewide student performance to state policy makers. Both states have 
policies in this area, but clearly, their policies differ in intensity. 
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Types of information disseminated. In 1993, 43 of the states (84 percent) had accountability policies 
p related to student test performance, requiring the reporting and dissemination of some form of student 

test-performance measures. Accountability policies related to nontest performance were even more 
prevalent. All but one state required the reporting and dissemination of nontest performance measures, 
such as school dropout rates, graduation rates, average daily attendance, and the proportion of students 
going to college. 

9 Audiences to whom performance information was disseminated. Most of the states widely 

disseminated performance reports in 1993. The general public was a major audience: when information 
was disseminated, more than seven-eighths of these states disseminated it to the general public. As 
expected, state-level information was disseminated to state policy makers, and district-level scores were 
disseminated to all districts in all states with this type of reporting. 

® Performance standards. Of the 43 states with accountability policies for student test performance in 

1993, most states (54 percent) required only the reporting and dissemination of student performance 
data, but they did not set standards for performance. Standards for test performance were specified by 
20 states; standards for nontest performance were specified by 15 states. The most prevalent type of 
sanction related to poor performance was negative publicity; the most prevalent type of reward was 
0 “official recognition/publicity.” Accreditation (or loss of it) was the next most common incentive and 

sanction, following official recognition/publicity. 

Testing Policies 

Testing policies provide states with an opportunity to influence what is taught in the schools. Policies 
® that link teacher and administrator rewards and sanctions to student test performance (“high-stakes 

testing”) can exert strong pressures to teach the curriculum covered by these tests. Since most state- 
mandated tests are aligned with the state's curricular frameworks, it is not surprising that policy activity 
in testing is strongly correlated with policy activity in curriculum. 

£ Between 1980 and 1993, the amount of state-mandated testing “increased” or “greatly increased” in 

two-thirds (67 percent) of the states, and the number of state-mandated or state-recommended uses of 
such tests had “increased” or “greatly increased” in three-quarters of the states (75 percent). 

High school subject areas. In 1993, most states required testing in mathematics (86 percent) and in 
reading (82 percent) in at least one high school grade. Approximately half of the states required testing 
^ in science (53 percent) and social studies (51 percent) in at least one high school grade; approximately 

one-third (37 percent) required testing in basic skills; approximately one-sixth (16 percent) in physical 
fitness; and only one state required testing in a foreign language. The greatest amount of state-mandated 
student performance assessment in mathematics, reading, science, history/social studies, and basic skills 
occurred in the 1 1th grade. 

^ High school alternative assessments. By 1993, more than two-thirds of the states (69 percent) had 

developed portfolio assessments for writing or were developing them. More than half (57 percent) had 
developed portfolio assessments for mathematics assessments or were developing them. 
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Curriculum and Instruction Policies 

Nearly half of the states (48 percent) changed their high school curricular policies in the period between 
1990 and 1993. In 1993, more than two-thirds of the states provided frameworks for high school 
students in each of the core subject areas (English, mathematics, and science). Most states also had 
developed or were developing curricular guidelines for instruction in nontraditional areas, such as work- 
place and life skills. 

The major motivators for using the state-provided curriculum are testing and the dissemination of test 
performance results. Almost two-thirds (63 percent) of the states that provided curricula in 1993 used 
them as the basis of mandated tests. Very few states ( 1 1 percent) deemed it necessary to provide 
financial incentives to schools to encourage the use of state curricular guidelines or learning 
competencies. Fewer still (4 percent) used financial incentives to encourage schools to provide a 
prescribed set of courses in core academic areas. 

Course requirements. In 1993, a majority of the states (61 percent) prescribed a set of required courses 
for high school students. Almost two-thirds of the states had high school graduation requirements in 
English equal to four-year state college admission requirements. One-third of the states had similar 
requirements in mathematics; 47 percent, in science. 

Classroom instructional policies. In 1993, five states had policies concerning homogeneous grouping 
(i.e., grouping students together according to their ability levels). All of these policies, which were 
implemented between 1985 and 1992, discouraged such grouping. 

Textbook policies. In 1993, more than one-third (37 percent) of the states had policies concerning 
school textbooks. Nine states specified which books were to be used in required courses; six states 
provided incentives for using approved textbooks. 

Public School Choice and Enrollment Policies 

The underlying goal of choice policies is to provide equal access to desirable public schools (such as a 
school with distinctive characteristics or educational approaches, or a school whose student test scores 
are high) to all students within a district or state. State policies related to choice and enrollment were 
quite stable during the 1980-1993 period. Most states deferred to local authorities to develop and 
implement public school choice plans. 

State policy activity in the area of high school choice and enrollment occurred in 17 states in 1993 and 
was independent of activity in all other policy areas except school organization. In 13 states, policies 
explicitly encouraged within-district open enrollment for high school students. In general, policies in 
public school choice and enrollment, whether dealing with inter- or intra-district choice, tended to be 
permissive rather than prescriptive or prohibitive. Districts were generally allowed to adopt and 
implement their own public school choice policies. 

Teacher Policies 

Policies dealing with certifying, hiring, and evaluating teachers have been one of the traditional ways 
for states to influence the delivery of education services. By specifying stringent standards, rigorous 
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training programs, continuous performance monitoring and performance-based rewards and sanctions, 
and incentives for professional growth and development, states can help ensure that high quality 
teachers are hired and that they are motivated to perform at the highest levels. Activity in the area of 
teacher policies was found to be significantly associated with the proportion of education funding that 
the state government controlled. The greater the proportion, the more the state implemented policies in 
this domain to affect education outputs. 

Entry requirements. In 1993, at least 44 states required high school teachers to have completed a 
specified number of credit hours in education or an academic area. Most states also required beginning 
teachers to meet certain initial certification criteria, pass an entrance test, and complete student teaching. 

Maintenance of quality. In 1993, more than three-quarters (78 percent) of the states required teachers 
to obtain additional formal education or in-service training to maintain certification, and 36 states 
required teachers with probationary certification to be evaluated. States also provided funds to districts 
or schools for in-service training (75 percent), supported teacher mentoring programs (73 percent), and 
sponsored in-depth conferences for high school teachers (67 percent). In 1993, more than half (59 
percent) of the states also mandated evaluation to make decisions about teacher retention or dismissal. 

School Organization 

School organization concerns the roles that different individuals play within a school, particularly with 
respect to decision making. State policy activity in this area is strongly correlated with policy activity in 
the area of performance reporting and dissemination (accountability). 

School-based decision making. Compared with 1 980 policies, newer policies in two-thirds (67 
percent) of the states “encouraged” or “strongly encouraged” school-based decision making (regarding 
such issues as curriculum, allocation of resources, facilities maintenance, and the hiring and firing of 
the principal or teachers). In spite of this strong trend towards decentralization, very few states (five or 
fewer) required school-based decision making in the areas investigated: curriculum, resource allocation, 
facilities maintenance, hiring and firing of the principal, and hiring and firing of teachers. 

School councils. Nearly all (90 percent) states reported that school councils were involved in school- 
level decision-making processes. However, most states either did not specify a role (44 percent) for 
them, or encouraged a primarily advisory role (37 percent). 

Other state school organization policies. In 1993, nearly half (47 percent) of the states had policies 
related to the size of classes in public schools, and 58 percent of these states decreased their target class 
sizes. Fewer than half (43 percent) of the states had policies regarding staffing-ratios (i.e., the number 
of students per professional). Between 1980 and 1993, seven states decreased their staffing ratios, and 
six states increased them. Finally, by 1993, 45 states either had a technology plan for their schools or 
were developing one. 

Accountability policy activity and school organization (particularly school-based management) policy 
activity were the policy areas that were selected for further analyses. Scores representing the level of 
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policy activity in each of the areas of interest were available from data files created for this purpose 
(Levine, 1996b). 

D. Analytic Procedures 

Within-school measures of organizational change were calculated for schools in the overlap sample, 
using items from the 1990-91 and 1993-94 SASS School Administrator and School Questionnaires. In 
these analyses, data were weighted using school weights from the 1993-94 SASS data files. These 
enabled generalization of findings to schools that existed at the time that the sampling frames for the 
1990-91 and 1993-94 SASS surveys were developed. 

One complication that arose in our analyses was that the primary measures of school organization 
available in the SASS data set were attitude and perception measures completed by the school principal. 
Schools that changed principals over this time period were identified, to enable some analyses to control 
for the potential effects of changes in perception due to a different respondent. In about 46 percent of 
the sample schools, the principal who completed the SASS survey in 1990-91 (the first time point in 
our two-wave panel) was no longer serving as principal in the spring of 1993-94, when the next SASS 
wave was conducted. 

Another complication concerned the potential unreliability in the available measures of organization 
and context. Unreliability in the measures will reduce the observed correlations in measures over time, 
potentially exaggerating the degree of instability over time in organizational features. To examine this 
issue, reliability coefficients (Cronbach’s alpha) for each organizational measure were calculated. Over- 
time correlations were compared with the scale reliabilities. 

Analyses of Longitudinal Change, Cross-tabulated by School Characteristics 



Analyses of ( 1 ) the influence of various groups in decision making and (2) the level to which various 
factors were perceived as problems by the principal were cross-tabulated by selected independent 
variables (urbanicity, school size, percent minority enrollment, school level, and whether or not the 
principal had changed over this time period). 

Schools were also categorized as showing either improvements (positive changes) or declines (negative 
changes) in their school climate. The principal’s responses to the school climate items (i.e., items asking 
the extent to which various factors were a problem in the school) in 1993-94 were compared with 
responses in 1990-91. Schools in which the response to these items were higher in 1993-94 were 
labeled as showing improvements in climate; those with decreases, declines in climate. Similarly, 
schools were categorized as having changes in the influence of various actors (positive or negative) in 
decision making, according to the principal report. The principal's response to the influence items in 
j 993 -94 were compiled with responses in 1990-91. Schools in which the response to a group s 
influence item was higher in 1993-94 were labeled as showing a positive influence for the group; those 
with declines, a negative influence. 

Changes in twelfth grade graduation rates and the proportion of graduates going on to higher education 
were calculated for each school with a twelfth grade. These changes were cross-tabulated with several 
of the independent variables discussed above (urbanicity, school size, minority enrollment, and whether 
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or not the school changed principals over this time period). For all of these analyses, both mean change 
scores and the proportion of schools that had changed were calculated. These data are presented in 
Table 1 and Tables 3-15. 

Finally, the percentage of schools that changed principals between 1990-91 and 1993-94 was 
estimated, overall and by several of the independent variables discussed above (urbanicity, school size, 
minority enrollment, school level, and region). Results are presented in Table 2. 

Analyses to Investigate Associations Between Changes in the Decision-making Practices and 
State Education Policy Activity 

To investigate whether or not recent reform-related policies enacted by education departments of the 
states have an influence on schools, a series of multivariate analyses was performed. This was done by 
merging the state education agency policy data, describing policy in 1992-93 (Levine and Huber man, 
1996), with the longitudinal school change data derived from the 1990-91 and 1993-94 SASS School 
Administrator Questionnaire items dealing with influence of various groups on decision making. The 
basic question examined was: Does educational policy variation among the state policies have an effect 
on variation among schools in changes in their decision-making processes from 1990-91 to 1993-94? 

Figure 2 shows the analytic model used for this analysis. For each school in the longitudinal file, a 
dependent variable measured in the recent wave (1994) was regressed on the same variable measured in 
the earlier wave (1991), plus the state policy of interest, plus seven control variables. These other 
variables controlled for different school characteristics which might be related to organizational 
decision-making measures (urbanicity, region, enrollment size, percent minority students, school 
climate, whether the school changed principals over this time, and influence level in 1990-91). By 
controlling for the effect of the level of the dependent variable (influence level) at time one, this 
straightforward longitudinal model shows the effect that any other independent variable might have on 
change from 1991 to 1994 (Tuma and Hannan 1984). 



Figure 2. Analysis model for examining change in school decision making 
Yjj = ccY T1 + P,X, + P,X, .... + p o 

where: Y=mean influence on school decisions over curriculum, student discipline policies, 

and faculty hiring 

Tl=1991 

T2=1994 

X,=State policies 1992 
Xi=Other school characteristics 



Since organizational decision making was going to be analyzed, two state policy domains that would be 
expected to influence this decision making were identified. The first type related to the use of student 
test scores for school accountability. The second was a policy that attempts to foster and increase more 
school-based management (SBM). SBM policies also included several other policy dimensions such as 
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the degree to which the SBM policy was integrated within the rest of the state department of 
education’s school management policies and how recent the SBM policy was put into action in the 
state. 

There were five dependent variables, each representing the average influence (on a six point scale) a 
different governance actor has across the school's decisions in curriculum policy, teacher hiring policies, 
and student discipline policies. The actors included the state department of education, school board, 
principal, teachers, and the parent association. Higher values indicate greater levels of influence exerted 
by the actor on these decisions. These variables, as previously indicated, were collected from principals’ 
perceptions of the decision-making environment that their schools function within. Results of these 
analyses are presented in Tables 16 and 17. 

E. Results and Discussion 

Gross Structural Change Analyses 

Excluding schools that added or dropped prekindergarten programs, 83 percent of the longitudinal 
schools had the same grade structures in 1990-91 and 1993-94. Changes in grade structure were 
apparently implemented in response to changing demographics and school policy. 

Some longitudinal schools underwent a major shift in the grade levels they served. Using longitudinal 
data, it is possible to estimate how many have done this and how substantial the changes were. Changes 
in the organizational structure of a school that underwent a major shift in grade levels served probably 
reflected the impact of the grade level shift more than anything else. Schools that had such radical 
changes should probably be analyzed separately and excluded from further analyses. 

Accordingly, changes and stability in enrollments, faculty, and student/teacher ratios were examined in 
schools that had the same grade structures in 1990-91 and 1993-94 (see table 1). These changes 
reflected demographic changes during the given time period. 



Table 1. Changes in selected characteristics of schools with the same grade span in 



1990-91 and 1993-94 









Net 


% schools 


% schools 


% schools not 


Mean values 


1990-91 


1993-94 


change 


increasing 


decreasing 


changing 


Student enrollment 


532.5 


547.4 


14.8 


59.5 


39.9 


0.6 


Student/teacher ratio 


16.8 


17.1 


0.3 


44.9 


54.9 


0.2 


Minority student 
enrollment (percent) 


26.0 


28.0 


2.1 


54.1 


27.0 


19.0 


Minority teachers 
(percent) 


12.2 


12.2 


-.10 


29.4 


27.7 


42.9 


SOURCE; U.S. Department of Education, National Center for Education Statistics, 1990- 


-91 and 1993- 


94 Schools and 



Staffing Surveys, “Public School Questionnaire ” 

NOTE; Net change may not equal the difference between years and percentages may not sum to 100 percent due to rounding. 
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Analyses of Organizational Leadership Change 

Close to half (46 percent) of the schools in the United States changed their principals over the 1990-91 
to 1993-94 time period. Schools in the West region were more likely to have had a change in 
leadership compared to schools in other regions, especially in comparison with those in the Midwest 
region (54 percent versus 38 percent for Midwest), versus 45 percent for the South and 44 percent for 
the Northeast (see table 2). Similarly, combined schools were more likely to change principals than 
elementary or secondary schools (54 percent versus 44 percent). Accordingly, a variable indicating 
whether a school changed its principal was introduced in our multivariate analysis of principal reporting 
on decision making to control for the potential effects of different respondents. 



Table 2. Percent of schools which have changed principal between 1990-91 and 1993-94 



Total 46.0 



Urbanicity 




Urban 


47.5 


Suburban 


43.7 


Rural 


43.7 


School size 




<150 


47.6 


150-499 


44.1 


500-749 


41.8 


750+ 


47.9 


Minority enrollment 




<20% 


44.2 


20%+ 


48.7 


School level 




Elementary 


44.1 


Secondary 


44.3 


Combined 


54.4 


Region 




Northeast 


44.3 


Midwest 


38.1 


South 


44.5 


West 


54.4 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Survey, 
“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 
Questionnaire.” 
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These preliminary analyses also showed that principal succession was more likely in schools with 
minority enrollments in excess of 20 percent (49 percent versus 44 percent in other schools). 



Descriptive Analyses of Other School Characteristics and Analyses to Identify Patterns 
among Schools in Terms of Organizational Change and Stability 

During the three-year period between the 1990-91 and the 1993-94 school years, as previously noted, 
schools had relatively little change in their grade structure. They had more change in terms of leadership 
(see table 2) and their principal’s perception of influence over school policy by various groups (see 
tables 3-8). 



Table 3. Principal reporting of influence of various groups 


over school policy, by urbanicity 




change 


Urban 

% schools 
changing 


Suburban 

change % schools 

changing 


change 


Rural 

% schools 
changing 


Establishing Curriculum 

State Dept, of Education 


0.20 


61.2 


0.27 


62.2 


0.11 


65.3 


School board 


-0.09 


70.9 


-0.15 


72.2 


0.04 


73.4 


Principal 


0.31 


67.2 


0.14 


64.9 


0.07 


69.3 


Teacher 


0.35 


71.1 


0.31 


68.8 


0.27 


66.3 


Librarian 


0.18 


70.8 


0.09 


70.4 


0.10 


71.8 


Parent Association 


0.32 


68.3 


0.25 


64.5 


0.21 


68.9 


Hiring new full-time teachers 


School board 


-0.27 


76.5 


0.03 


75.0 


0.11 


73.3 


Principal 


0.25 


56.8 


0.03 


45.4 


0.02 


45.9 


Teacher 


0.76 


74.7 


0.76 


70.7 


0.75 


73.1 


Parent Association 


0.57 


57.9 


0.49 


53.1 


0.30 


50.5 


Setting discipline policy 


State Dept, of Education 


0.01 


81.7 


-0.01 


72.5 


-0.08 


75.3 


School board 


-0.09 


66.2 


0.02 


68.5 


0.13 


68.0 


Principal 


0.20 


62.6 


0.09 


51.6 


0.04 


52.5 


Teacher 


0.28 


68.8 


0.34 


61.7 


0.25 


65.0 


Parent Association 


0.46 


74.6 


0.46 


73.0 


0.34 


73.1 



bUUKLh: U.5. ue panmem oi c.uuwauuii, l'lauuuai 1 ° 

“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 

Questionnaire.” 
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Table 4. Principal reporting of influence of various groups over 


school policy, by school size 




<150 


150 


499 


500-749 




750+ 




change 


% 

schools 


change 


% 

schools 


change 


% 

schools 


change 


% 

schools 






changing 




changing 




changing 




changing 


Establishing Curriculum 
















State Dept, of 
Education 


0.03 


74.9 


0.09 


62.2 


0.38 


64.1 


0.23 


59.8 


School board 


-0.13 


74.3 


-0.01 


71.2 


-0.08 


72.9 


-0.02 


74.8 


Principal 


-0.06 


69.5 


0.15 


67.1 


0.11 


65.1 


0.26 


71.4 


Teacher 


0.30 


66.1 


0.31 


67.4 


0.26 


67.2 


0.32 


72.5 


Librarian 


0.34 


66.0 


0.03 


73.3 


0.12 


67.8 


0.20 


72.9 


Parent Association 


0.18 


65.2 


0.26 


66.4 


0.15 


66.4 


0.38 


74.1 


Hiring new full-time 


teachers 
















School board 


0.19 


69.9 


-0.04 


74.2 


-0.06 


76.1 


0.10 


75.7 


Principal 


0.14 


48.8 


0.07 


49.3 


-0.00 


46.9 


0.14 


46.3 


Teacher 


0.71 


67.9 


0.79 


71.8 


0.66 


75.4 


0.82 


75.3 


Parent Association 


0.39 


46.3 


0.37 


51.2 


0.39 


55.7 


0.55 


57.5 


Setting discipline policy 
















State Dept, of 
Education 


0.03 


72.1 


-0.11 


77.9 


0.14 


72.4 


-0.12 


77.7 


School board 


0.14 


71.6 


0.10 


68.6 


0.01 


65.8 


-0.08 


65.6 


Principal 


0.23 


61.4 


0.08 


53.3 


0.08 


53.2 


0.07 


55.8 


Teacher 


0.49 


69.5 


0.28 


63.2 


0.20 


65.6 


0.30 


66.3 


Parent Association 


0.52 


74.9 


0.35 


71.0 


0.34 


77.0 


0.54 


74.8 



SOURCE: U.S. Department of Education National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, 
“Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 5. 



Principal reporting of influence of various groups over school policy, by percent 
minority enrollment 



less than 20% minority 

change % schools changing 



20% minority and over 

change % schools changing 



Establishing Curriculum 


State Dept, of Education 


0.13 


School board 


-0.04 


Principal 


0.11 


Teacher 


0.34 


Librarian 


0.08 


Parent Association 


0.24 


Hiring new full-time teachers 


School board 


0.08 


Principal 


0.04 


Teacher 


0.81 


Parent Association 


0.34 


Setting discipline policy 


State Dept, of Education 


-0.09 


School board 


0.09 


Principal 


0.11 


Teacher 


0.32 


Parent Association 


0.37 



64.6 


0.24 


62.1 


72.2 


-0.03 


72.9 


68.6 


0.18 


66.2 


67.1 


0.25 


69.4 


71.2 


0.16 


71.2 


65.9 


0.26 


69.8 



74.3 


-0.10 


74.6 


44.3 


0.13 


53.5 


71.1 


0.68 


75.2 


50.3 


0.50 


56.4 



75.9 


0.03 


76.0 


71.9 


0.00 


62.0 


52.2 


0.07 


57.7 


62.4 


0.24 


68.4 


71.2 


0.44 


76.5 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 6. Principal reporting of influence of various groups 
principal or not 


over school policy, by the same 


Same principal 


Not the same principal 


change % schools 


change % schools 


changing 


changing 



Establishing Curriculum 


State Department of Education 


0.19 


58.9 


0.16 


69.4 


School board 


-0.09 


67.8 


0.02 


78.4 


Principal 


0.10 


65.0 


0.19 


70.9 


Teacher 


0.28 


64.3 


0.33 


72.8 


Librarian 


0.05 


70.1 


0.19 


72.5 


Parent Association 


0.17 


62.8 


0.34 


73.5 


Hiring new full-time teachers 


School board 


-0.10 


70.0 


0.14 


80.0 


Principal 


0.06 


43.1 


0.09 


54.5 


Teacher 


0.64 


69.3 


0.90 


77.2 


Parent Association 


0.26 


46.3 


0.60 


60.9 


Setting discipline policy 


State Department of Education 


-0.14 


71.6 


0.09 


81.4 


School board 


0.05 


65.9 


0.05 


69.9 


Principal 


0.11 


50.5 


0.07 


59.5 


Teacher 


0.32 


60.9 


0.24 


70.0 


Parent Association 


0.35 


68.1 


0.46 


80.0 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Survey, 
“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 
Questionnaire.” 
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Table 7. Principal reporting of influence of various groups over school policy, by school level 

Elementary Secondary Combined 



change % schools change 


% schools change 


% schools 


changing 


changing 


changing 



Establishing Curriculum 



State Dept, of Education 


0.18 


63.0 


0.16 


65.0 


0.19 


63.9 


School board 


-0.03 


72.3 


-0.11 


73.6 


0.15 


69.8 


Principal 


0.20 


67.1 


0.04 


67.7 


-0.29 


76.4 


Teacher 


0.31 


66.6 


0.32 


71.3 


-0.03 


73.7 


Librarian 


0.13 


70.7 


0.12 


72.6 


-0.14 


71.0 


Parent Association 


0.24 


67.3 


0.27 


69.2 


0.12 


63.5 


Hiring new full-time teachers 


School board 


-0.04 


74.8 


0.04 


73.6 


0.50 


73.7 


Principal 


0.06 


49.4 


0.13 


44.4 


-0.02 


50.1 


Teacher 


0.76 


73.3 


0.73 


71.4 


0.84 


73.9 


Parent Association 


0.43 


53.3 


0.36 


51.2 


0.36 


59.1 


Setting discipline policy 


State Dept, of Education 


-0.05 


77.5 


-0.07 


71.4 


0.36 


76.9 


School board 


0.03 


68.0 


0.04 


66.1 


0.40 


72.7 


Principal 


0.09 


54.9 


0.13 


52.5 


-0.07 


59.4 


Teacher 


0.27 


63.9 


0.34 


67.0 


0.25 


70.0 


Parent Association 


0.36 


73.6 


0.49 


73.0 


0.49 


72.7 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 8. Principal reporting of influence of various groups over school policy in schools with 
reported changes in group’s i nfluence 

Positive Negative Overall 

% schools % schools % schools 





change 


changing 


change 


changing 


change 


changing 


Establishing Curriculum 


State Dept, of Education 


1.73 


35.9 


-1.61 


27.7 


0.18 


63.6 


School board 


1.72 


35.7 


-1.77 


36.8 


-0.04 


72.5 


Principal 


1.66 


37.1 


-1.57 


30.5 


0.14 


67.6 


Teacher 


1.68 


42.2 


-1.59 


25.8 


0.30 


68.1 


Librarian 


1.71 


38.7 


-1.69 


32.5 


0.11 


71.2 


Parent Association 


1.66 


40.2 


-1.54 


27.3 


0.24 


67.6 


Hiring new full-time teachers 


School board 


1.72 


38.2 


-1.77 


36.3 


0.00 


74.5 


Principal 


1.65 


24.6 


-1.41 


23.6 


0.07 


48.2 


Teacher 


2.02 


53.4 


-1.66 


19.4 


0.76 


72.8 


Parent Association 


1.85 


36.5 


-1.63 


16.4 


0.41 


52.8 


Setting discipline policy 


State Dept, of Education 


1.90 


36.8 


-1.88 


39.2 


-0.04 


76.0 


School board 


1.68 


36.0 


-1.74 


31.7 


0.05 


67.7 


Principal 


1.47 


29.3 


-1.35 


25.2 


0.09 


54.5 


Teacher 


1.56 


40.3 


-1.40 


24.6 


0.28 


64.9 


Parent Association 


1.89 


46.6 


-1.80 


26.8 


0.40 


73.4 



SOURCE: U.S. Department of Education, National Center for Education Statistics. 1990-91 Schools and Staffing Survey, 
“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Surveys, “Public School Principal 
Questionnaire.” 



Preliminary results on general school change indicate that, overall, the kinds of organizational 
characteristics that were measured were relatively stable features of schools, at least over the three-year 
period of our study. However, substantial changes in several other characteristics were observed, 
especially in the degree of influence of parents, teachers, and principals over decisions about curriculum 
and instruction (see tables 3-8). Furthermore, preliminary results indicate that change in organizational 
characteristics is related to school context. For example, an increase in teacher influence between 
1990-91 and 1993-1994 was more likely in some types of schools (e.g., schools with lower minority 
enrollment) than in others. 
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Total student enrollment (see table 1), student/teacher ratio (see table 1), high school graduation rate 
(see table 9), and college application rate (see table 9) stayed rather constant. Overall, the school 
environment had changed relatively little. The 1990-91 and 1993-94 SASS School Administrator 
Questionnaires asked principals to report on how serious 20 different problems were in their schools 
(see tables 10- 15). These problems included basic and behavior standards; involvement; attendance; 
and respect for teacher, students, and property. On a 4-point scale, the biggest changes were a reduction 
of .15 scale units for “physical conflicts among students” and declines of .10 scale units for “student 
possession of weapons” and “racial tension.” 



Table 9. Graduation and college application rate change, by school type 



Graduation Rate 



change 



% schools 
changing 



College Application Rate 

% schools 
changing 



change 



Total 


-2.11 


81.8 


1.23 


96.6 


Urbanicity 

Urban 


-4.50 


89.0 


0.57 


92.0 


Suburban 


-0.03 


93.1 


0.79 


98.5 


Rural 


-2.19 


76.1 


1.55 


97.2 


School size 
<150 


-5.09 


53.8 


8.03 


83.9 


150-499 


-1.46 


78.7 


-0.14 


99.2 


500-749 


-3.21 


89.9 


2.95 


99.8 


750+ 


-0.80 


94.8 


2.95 


99.6 


Minority enrollment 

<20% 


-1.88 


77.9 


1.56 


97.6 


20%+ 


-2.57 


88.8 


0.58 


95.0 


Leadership 
Same principal 


-0.53 


81.3 


1.57 


96.9 


Not same principal 


-4.70 


81.5 


0.50 


97.4 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 10. Principal reporting of s chool environment, by urbanicity 

Urban Suburban Rural 





change 


% schools 
changing 


change 


% schools 
changing 


change 


% schools 
changing 


Teacher absenteeism 


-0.01 


50.2 


0.04 


43.5 


-0.00 


45.5 


Student tardiness 


-0.03 


53.4 


-0.00 


50.8 


-0.06 


49.1 


Student absenteeism 


0.00 


56.3 


0.03 


43.8 


-0.01 


50.3 


Student cutting classes 


-0.03 


25.7 


-0.03 


21.9 


-0.06 


24.2 


Student dropping out 


0.05 


24.2 


0.04 


16.9 


0.02 


25.7 


Student apathy 


-0.05 


53.1 


-0.11 


47.1 


-0.07 


55.2 


Physical conflicts among 


-0.22 


53.5 


-0.07 


41.0 


-0.17 


49.2 


students 


Robbery or theft 


-0.06 


45.5 


-0.03 


38.9 


-0.09 


38.5 


Vandalism of school property 


-0.05 


53.1 


-0.02 


39.7 


-0.09 


40.2 


Student pregnancy 


-0.05 


18.0 


0.01 


15.9 


0.00 


25.5 


Student use of alcohol 


0.05 


21.0 


0.04 


19.9 


0.04 


32.6 


Student drug abuse 


-0.00 


21.6 


0.00 


23.3 


0.01 


29.6 


Student possession of 


-0.10 


34.5 


-0.08 


26.9 


-0.11 


22.9 


weapons 


Student disrespect for 


-0.07 


47.7 


-0.00 


47.2 


“0.04 


48.9 


teachers 


Verbal abuse of teachers 


-0.12 


47.7 


0.07 


46.9 


-0.04 


46.0 


Lack of academic challenge 


0.02 


48.1 


-0.03 


47.0 


0.06 


53.3 


Lack of parent involvement 


0.02 


51.6 


0.09 


56.6 


0.08 


58.1 


Parental alcoholism and/or 


- 0.11 


51.9 


-0.00 


48.7 


-0.00 


50.0 


drug abuse 


Poverty 


-0.13 


45.7 


-0.13 


46.8 


-0.05 


45.5 


Racial tension 


-0.12 


42.4 


-0.10 


38.5 


-0.09 


32.1 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Survey, 
“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 
Questionnaire.” 
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Table 11. Principal reporting of school environment, by school size 





<150 

% 

change schools 

changing 


150- 

change 


499 

% 

schools 

changing 


500- 

change 


749 

% 

schools 

changing 


750+ 

% 

change schools 

changing 


Teacher absenteeism 


0.02 


35.2 


-0.02 


45.2 


0.00 


45.5 


0.06 


55.5 


Student tardiness 


-0.13 


48.7 


-0.01 


49.3 


-0.07 


48.7 


-0.02 


57.9 


Student absenteeism 


0.00 


56.0 


0.02 


48.4 


-0.04 


50.9 


0.00 


49.0 


Student cutting classes 


0.07 


24.9 


-0.07 


20.1 


-0.05 


21.7 


-0.06 


37.6 


Student dropping out 


0.11 


29.5 


-0.01 


18.0 


0.03 


20.7 


0.09 


36.9 


Student apathy 


0.02 


57.6 


-0.09 


49.4 


-0.13 


55.1 


0.00 


55.1 


Physical conflicts 


-0.14 


44.0 


-0.14 


47.3 


-0.19 


45.4 


-0.15 


55.7 


among students 


Robbery or theft 


-0.10 


35.9 


-0.06 


36.1 


-0.01 


45.3 


-0.14 


47.4 


Vandalism of school 


-0.09 


40.5 


-0.04 


41.7 


-0.03 


44.1 


-0.14 


46.6 


property 


Student pregnancy 


-0.02 


30.8 


0.01 


16.1 


-0.00 


20.0 


-0.06 


32.4 


Student use of alcohol 


0.13 


40.9 


-0.00 


21.3 


0.06 


24.9 


0.07 


35.7 


Student drug abuse 


0.09 


34.8 


-0.02 


22.0 


0.01 


25.3 


-0.00 


34.0 


Student possession of 


-0.05 


19.8 


-0.09 


20.0 


-0.11 


31.4 


-0.17 


43.3 


weapons 


Student disrespect for 


-0 09 


45.8 


-0.01 


48.7 


-0.07 


45.2 


-0.02 


52.4 


teachers 


Verbal abuse of 


0.05 


45.8 


-0.01 


45.4 


-0.08 


48.7 


-0.07 


47.8 


teachers 


Lack of academic 


-0.10 


52.3 


0.03 


50.4 


0.05 


49.9 


0.08 


50.1 


challenge 


Lack of parent 


003 


59.5 


0.02 


55.5 


0.09 


55.8 


0.21 


57.1 


involvement 


Parental alcoholism 


0.05 


49.6 


-0.03 


50.0 


-0.06 


48.1 


-0.04 


53.2 


and/or drug abuse 


Poverty 


-0.05 


51.0 


-0.10 


45.8 


-0.10 


43.5 


-0.05 


46.7 


Racial tension 


-0.06 


22.2 


-0.09 


32.6 


-0.09 


40.0 


-0.17 


49.8 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 12. Principal reporting of school environment, by percent minority enrollment 



less than 20% minority 



change 



% schools 
changing 



20% minority or more 



, % schools 

change , 

changing 



Teacher absenteeism 


0.03 


44.5 


-0.03 


48.0 


Student tardiness 


-0.03 


49.9 


-0.05 


51.4 


Student absenteeism 


0.02 


47.6 


-0.02 


53.1 


Student cutting classes 


-0.04 


21.7 


-0.05 


27.0 


Student dropping out 


0.02 


20.5 


0.05 


26.3 


Student apathy 


-0.04 


51.5 


-0.12 


54.0 


Physical conflicts among students 


-0.13 


45.0 


-0.19 


52.0 


Robbery or theft 


-0.08 


35.4 


-0.05 


46.7 


Vandalism of school property 


-0.05 


40.3 


-0.07 


46.6 


Student pregnancy 


-0.00 


20.2 


-0.02 


22.7 


Student use of alcohol 


0.03 


27.1 


0.05 


25.9 


Student drug abuse 


0.01 


26.0 


-0.01 


26.2 


Student possession of weapons 


-0.09 


20.5 


-0.12 


34.8 


Student disrespect for teachers 


-0.00 


47.9 


-0.09 


48.6 


Verbal abuse of teachers 


0.02 


45.0 


-0.11 


48.8 


Lack of academic challenge 


0.02 


49.1 


0.04 


52.3 


Lack of parent involvement 


0.08 


56.1 


0.06 


56.5 


Parental alcoholism and/or drug 
abuse 


0.01 


48.0 


-0.08 


52.9 


Poverty 


-0.09 


44.3 


-0.09 


48.1 


Racial tension 


-0.10 


28.5 


-0.09 


46.6 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 13. Principal reporting of school environment, by the same principal or not 



change 



Same principal 

% schools 



Not the same principal 

, % schools 

change , hai , oino 



Teacher absenteeism 


-0.05 


42.2 


0.07 


50.7 


Student tardiness 


-0.03 


45.5 


-0.05 


56.8 


Student absenteeism 


-0.01 


43.5 


0.02 


57.8 


Student cutting classes 


-0.05 


20.0 


-0.05 


28.9 


Student dropping out 


0.01 


19.2 


0.05 


27.6 


Student apathy 


-0.10 


47.3 


-0.04 


59.0 


Physical conflicts among students 


0.17 


45.2 


-0.13 


51.3 


Robbery or theft 


-0.10 


34.4 


-0.02 


47.4 


Vandalism of school property 


-0.07 


38.7 


-0.05 


48.2 


Student pregnancy 


-0.00 


17.7 


-0.02 


25.6 


Student use of alcohol 


0.04 


22.8 


0.03 


31.3 


Student drug abuse 


-0.01 


23.7 


0.02 


29.1 


Student possession of weapons 


-0.11 


22.4 


-0.09 


31.7 


Student disrespect for teachers 


-0.06 


43.1 


-0.01 


54.5 


Verbal abuse of teachers 


-0.03 


42.4 


-0.04 


51.9 


Lack of academic challenge 


0.04 


44.2 


0.02 


58.2 


Lack of parent involvement 


0.04 


51.2 


0.11 


62.6 


Parental alcoholism and/or drug abuse 


-0.06 


43.7 


0.02 


58.0 


Poverty 


-0.08 


38.9 


-0.10 


54.6 


Racial tension 


-0.12 


30.8 


-0.07 


42.7 



“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 
Questionnaire.” 



35 A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 



o 

ERLC 



45 



Table 14 . Principal reporting of school environment, by school level 



Elementary Secondary Combined 





change 


% schools 
changing 


change 


% schools 
changing 


change 


% schools 
changing 


Teacher absenteeism 


0.01 


44.1 


0.03 


50.5 


-0.15 


49.5 


Student tardiness 


-0.05 


49.5 


0.01 


53.0 


-0.22 


52.6 


Student absenteeism 


0.01 


48.3 


0.02 


54.2 


-0.15 


50.4 


Student cutting classes 


-0.02 


15.2 


-0.10 


45.8 


-0.19 


38.1 


Student dropping out 


0.03 


14.0 


0.05 


44.2 


-0.12 


43.6 


Student apathy 


-0.09 


51.4 


-0.07 


55.5 


0.08 


53.6 


Physical conflicts among students 


-0.12 


47.2 


-0.23 


48.9 ' 


-0.26 


53.7 


Robbery or theft 


-0.03 


37.2 


-0.14 


46.7 


-0.22 


49.7 


Vandalism of school property 


-0.05 


42.1 


-0.07 


45.2 


-0.13 


43.1 


Student pregnancy 


0.02 


10.3 


-0.09 


47.2 


0.03 


46.8 


Student use of alcohol 


0.03 


15.4 


0.07 


54.0 


0.02 


46.5 


Student drug abuse 


0.01 


17.8 


-0.03 


46.5 


0.06 


40.3 


Student possession of weapons 


-0.06 


21.7 


-0.23 


41.2 


-0.12 


19.3 


Student disrespect for teachers 


0.00 


47.0 


-0.13 


49.6 


-0.11 


59.3 


Verbal abuse of teachers 


0.02 


46.0 


-0.16 


48.6 


-0.16 


45.0 


Lack of academic challenge 


0.04 


49.0 


-0.01 


53.2 


0.13 


57.5 


Lack of parent involvement 


0.11 


56.5 


-0.01 


56.5 


-0.10 


51.9 


Parental alcoholism and/or drug 
abuse 


-0.02 


48.0 


-0.06 


54.6 


-0.05 


55.7 


Poverty 


-0.09 


46.1 


-0.07 


44.8 


-0.19 


49.2 


Racial tension 


-0.06 


34.3 


-0.20 


39.1 


-0.04 


47.8 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Surveys, 
“Public School Questionnaire” and “Public School Administrator Questionnaire” and 1993-94 Schools and Staffing 
Surveys, “Public School Questionnaire” and “Public School Principal Questionnaire.” 
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Table 15. Principal reporting of school environment in schools with reported changes in school 
environment 



Positive Negative Overall 





change 


% schools 
changing 


change 


% schools 
changing 


change 


% schools 
changing 


Teacher absenteeism 


1.18 


22.8 


-1.14 


23.2 


0.01 


46.0 


Student tardiness 


1.15 


23.2 


-1.11 


27.4 


-0.04 


50.5 


Student absenteeism 


1.17 


24.4 


-1.11 


25.5 


0.00 


49.9 


Student cutting classes 


1.14 


9.6 


-1.08 


14.3 


-0.05 


23.9 


Student dropping out 


1.24 


12.1 


-1.11 


10.9 


0.03 


23.0 


Student apathy 


1.14 


24.1 


-1.23 


28.5 


-0.08 


52.5 


Physical conflicts among students 


1.14 


17.2 


- 1.14 


30.8 


-0.15 


47.9 


Robbery or theft 


1.07 


17.3 


-1.10 


22.9 


-0.07 


40.2 


Vandalism of school property 


1.07 


19.2 


-1.12 


23.7 


-0.06 


42.9 


Student pregnancy 


1.15 


10.3 


-1.14 


11.0 


-0.01 


21.2 


Student use of alcohol 


1.24 


14.5 


-1.17 


12.1 


0.04 


26.6 


Student drug abuse 


1.15 


12.8 


-1.09 


13.2 


0.00 


26.1 


Student possession of weapons 


1.10 


8.4 


-1.08 


18.2 


-0.10 


26.6 


Student disrespect for teachers 


1.14 


22.5 


-1.15 


25.7 


-0.04 


48.2 


Verbal abuse of teachers 


1.13 


21.9 


-1.13 


24.8 


-0.03 


46.6 


Lack of academic challenge 


1.20 


26.6 


-1.22 


23.9 


0.03 


50.4 


Lack of parent involvement 


1.23 


30.4 


-1.18 


25.9 


0.07 


56.3 


Parental alcoholism and/or drug 
abuse 


1.21 


23.7 


-1.20 


26.3 


-0.03 


50.0 


Poverty 


1.19 


19.1 


-1.18 


26.8 


-0.09 


45.9 


Racial tension 


1.14 


14.2 


-1.18 


22.0 


-0.10 


36.1 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 Schools and Staffing Survey, 
“Public School Administrator Questionnaire” and 1993-94 Schools and Staffing Survey, “Public School Principal 
Questionnaire.” 



Compared to their perception of school environment, principals’ perception of influence over school 
policy by various groups (state department of education, school board, principal, teacher, and parent 
association) has changed relatively more during the same time period. Principals reported that teachers 
and parent associations have more influence over school policies regarding establishing curriculum, 
hiring new full-time teachers, and setting school policy. 

These preliminary results suggest that the collection of data on school organization for a longitudinal 
sample can contribute to our understanding of school organization and school change, and that 
important conclusions can be drawn from panel data that cannot be derived from trend data alone. 
Further work is needed to identify the most promising school organization measures to include in future 
panel studies and to determine the most appropriate sampling plan. In particular, attention must be 
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given to the appropriate time interval between data collections in future panel studies, and the number 
of periods over which data should be collected for each school in the sample. 

Analyses to Investigate Associations Between Changes in the Decision-making Practices and 
State Education Policy Activity 

Recent research has given considerable attention to the importance of school organizational 
characteristics in explaining teacher effectiveness and student outcomes (e.g., Hallinan, 1995). Work in 
this tradition has focused on a diverse set of organizational features, including the structure of the 
curriculum, teacher work conditions, the locus of decision making, the school climate, and the degree of 
consensus on goals and methods of instruction. While work on school organization holds promise in 
identifying school features associated with achievement, little is known about the ways school 
organizational features develop over time. In particular, almost no data are available to determine 
whether organizational features (such as the relative influence of various organizational actors on key 
decisions) are relatively stable features of schools, changing over a long time period, or whether they 
are relatively transient features, adjusting with changes in instructional and administrative staff or 
student composition. It also is not clear whether changes in organizational characteristics are more 
likely in some types of schools than others. 

Most previous longitudinal studies, such as High School and Beyond and NELS:88, have focused on 
students as the primary unit of the analysis. However, with the exception of a recent follow-up study of 
High School and Beyond Schools (Levine, 1996), there is almost no analysis of school change with a 
large nationally representative sample. 

Data from the SASS Administrator and School Questionnaires were used to estimate the amount of 
organizational change that had occurred in each of the schools in the overlap sample of public schools. 
The extent to which the degree of observed change in school decision-making processes differed across 
states that have taken different approaches to educational reform was one of the foci of these 
examinations. The past two decades have witnessed a large volume of state policy-making activity in 
education, and different states have employed different strategies to encourage school improvement. 

One key dimension of variation across states is the degree of emphasis placed on school-based decision 
making. Some states like Hawaii and Colorado have given considerable emphasis to encouraging 
school-based management, while other states like Michigan and Virginia have given these policies 
much less attention. States also vary in the emphasis given to accountability policies, policies 
encouraging or requiring schools to report routinely on student achievement, graduation rates, and other 
key outcomes. 

These two reform strategies might be expected to have different effects on school organization. 
Stevenson and Schiller (1997), for example, argue that school-based management is an “exhortatory 
policy,” based on professional images of the form a reform-oriented school should take. One might 
expect such policies to diffuse through professional networks. Accountability policies, on the other 
hand, involve an emphasis on school outputs rather than procedures. The effects of accountability 
policies on school organization are difficult to predict. Hence, as Stevenson and Schiller argue, we 
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should expect stronger, more consistent effects of statewide school-based management policies on 
school organization than statewide accountability policies. 

Table 16 presents a summary of the effects of state policies from the twenty-five ordinary least squares 
(OLS) regressions. There are several major conclusions from this analysis about the effects of state 
policies on changes in school decision-making processes. 



Table 16. Summary of e ffects of state policies on changes in school decision making: 1991-1994 

State Policies 



Influence on 
school decisions 


Accountability 


School-based 

management 

policy 


School-based 

management 

integrated 

policy 


Recent school- 
based 

management 

policy 


SBM: 

interaction 

effect 


State Dept, 
of Education 


+ 


+ 


+ 


+ 


N.S. 


District School 
Board 


N.S. 


- 


- 


- 


N.S. 


Principal 


N.S. 


N.S. 


N.S. 


N.S. 


N.S. 


Teacher 


N.S. 


N.S. 


+ 


+ 


+ 


Parent Association 


N.S. 


N.S. 


+ 


+ 


N.S. 


R 2 range 


.11-25 


.11 -.24 


.11 — .26 


.11 -.24 


.11- .24 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 and 1993-94 Schools and 
Staffing Surveys, “School Administrator Questionnaire,”and Levine, R., “Changes in State Education Policies, 1980- 1993. 
The National Longitudinal Study of Schools. Palo Alto: American Institutes for Research, 1997. 

NOTE: Cell entries are OLS coefficients; + means “positive significant”; — means “negative significant”; N.S. means “not 
significant”. 

First, as predicted, accountability policy intensifies the tendency for state departments of education to 
have greater impact upon school’s managerial decisions. However, accountability polices have no 
effects on the rest of the actors studied in the education decision-making process. These kinds of state 
policies are not associated with any changes in the level of influence of school boards, principals, 
teachers, or parent associations on school operations. Accountability policies increase school’s 
sensitivity to state governance but not to other stakeholders in the education system. In a sense, 
accountability tightens one of the usually loosely coupled links between authority and the organizational 
production in education, but it has little influence on other links. 

Second, the same is almost true for SBM policies. If a state has an explicit SBM policy, schools tend to 
see an increase in the influence of state department of education on their day-to-day operations. The 
irony of this finding is discussed below. However, unlike the effects of accountability policies, state 
SBM policies tend to lower the influence of district school boards on school decision-making. Further, 
contrary to what might be predicted about policies that attempt to increase school level actors’ influence 
on decision making, a state SBM policy did not have an effect on the influence of principals, teachers, 
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or parent associations. In the case of principals, this may be a technical limitation of the dependent 
measure. Principals are often so heavily involved in school decisions, the scale of the measurement may 
not allow enough “ceiling” to capture changes in principals’ even greater influence. As shown in table 
17, on the six-point influence scale 6 , mean principals’ influence was already high in 1991 and rose even 
more by 1 994. 



6 On the 1990-91 SASS School Administrator Questionnaire, respondents were instructed to indicate the amount of 
influence each group or person has on decisions concerning various activities, “using the scale 1-6, where 1 is ‘None’ and 6 
is ‘A great deal.’” 



A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 



41 



Table 17. State policy scores and decision makers’ influence scores: means and standard 
deviations 



Mean Standard 

deviation 



State policies 



Accountability 


9.75 


4.12 


School-based management policy 


0.45 


0.50 


School-based management integrated policy 


7.85 


5.35 


Recent school-based management policy 


0.28 


0.45 


Influence 






Department of Education 1 99 1 


3.67 


1.22 


Department of Education 1994 


3.74 


1.17 


School Board 1991 


4.23 


1.11 


School Board 1994 


4.20 


1.08 


Principal 1991 


4.97 


0.91 


Principal 1994 


5.08 


0.82 


Teacher 1991 


3.97 


1.09 


Teacher 1994 


4.40 


1.06 


Parent Association 1991 


2.28 


0.99 


Parent Association 1994 


2.63 


1.08 



SOURCE: U.S. Department of Education, National Center for Education Statistics, 1990-91 and 1993-94 Schools and 
Staffing Surveys, “School Administrator Questionnaire,”and Levine, R., “Changes in State Education Policies, 1980- 1993: 
The National Longitudinal Study of Schools. Palo Alto: American Institutes for Research, 1997. 

NOTE: For information about how state policy scores were calculated, see Levine (1997a). Higher influence scores indicate 
greater amounts of influence. 



Why should a state SBM policy not increase the decision-making power of teachers and parents 
groups? To examine this further an index that measured the degree to which SBM policies were 
integrated into a State’s approach to the management of schools was constructed. States can just 
proclaim a SBM policy and not connect it to other policies that bring it into practice, or states can take a 
more integrated approach to SBM. Or, states may not even have an explicitly stated SBM policy, but 
may have adapted other related policy that fits a general SBM approach. The index captured a wider 
array of state policies that might have an impact on SBM. The effects of the variable are summarized in 
the third column (“School-based management integrated policy”) of table 16. Schools in states with an 
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integrated SBM policy still tended to perceive an increase in influence from the state department of 
education and a lowering of influence of the district administration over the three-year time span. But, 
more in line with the intentions of a SBM approach, both teachers and parent association groups gained 
influence over time in states with integrated policies. 

Third, since the state policy study examined when various policies were adopted by states, we can 
examine the effect of recent SBM policies versus older or no SBM policies. Since recent reforms have 
pushed SBM philosophies to such a large extent, we predicted that recent SBM policies might have 
more salient effects on changes in school decision-making. And the results indicate precisely that. 
Schools in states with recently adopted SBM policies have increased both teachers’ and parent 
association group’s influence on decisions and decreased the influence of the school board, (see Table 
16.) 

Fourth, to examine further the dynamics of governance policy and school change, we constructed an 
interaction term between the integrated SBM index and the timing of adoption of a SBM policy. States 
that have recent, integrative SBM policies expanded the influence of teachers in school decision 
making. And this model (not shown completely in figure 2) also indicates that after controlling for this 
joint effect, states which had just recently adopted an SBM policy, but without any integration of this 
policy into other management policies, had decreases in teacher influence on decision making. An 
interaction between an integrated policy and its timing only had an effect on teachers. 

Accountability policies had the predicted impact; namely, they increased the influence of the state 
department of education on schools. Surprisingly, SBM policies did this too. There is some irony in the 
finding that a policy intended to focus more decision-making involvement of the school results in 
greater influence of the state department of education. One way to think about this is to consider where 
most of the bureaucratization of schooling comes from in the American system of education and how a 
SBM policy would have an impact on that. The extensive local nature of schooling in the United States 
means that many administrative processes pile up at the school district level. And although there are 
some 15,000 school districts, many of these are very small, serving small populations of schools and 
students. Consequently, most schools are a part of the larger school districts in the nation. Since 
administrative size has long been known to be associated with more centralized management processes, 
the school district is likely the greatest obstacle to increasing decision-making power at the school level. 
This makes sense considering our findings that state SBM policies do decrease the influence of the 
school boards. And in doing that the state’s department of education may elevate their own influence on 
schools along with increasing the influence of teachers and parents. 

Another way to think about this ironic effect is that policies can often have unintended effects, 
particularly when they are not well integrated in administrative practices. If a state publicly and loudly 
proclaims its embrace of SBM principles, but does not do much to integrate them into actual practices, 
schools could perceive a greater influence of the state department but never reap the benefits of greater 
decision-making power. This is the difference between the effects we find for a stated policy only 
versus one that is integrated into a set of policies. 
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Summary and Conclusions 



Longitudinal data from the SASS overlap sample can be used to provide indications of how schools 
have changed. These data can be linked with exogenous sources to provide insights and knowledge that 
would not be otherwise possible through cross-sectional analyses. Since there are obvious potential 
benefits and few, if any, additional costs associated with the overlap sample, it is recommended that 
future SASS administrations include an overlap sample. 
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Comparison of Items on the SASS School 
Administrator Questionnaire 
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Data items comparable between 1987-88, 1990-91, and 1993-94 SASS 
Administrator Questionnaire (Differences shown in bold, na=not asked) 



Item topic 


1987-88 


1990-91 


1993-94 


BA/BS 


la. Which of the following 
college degrees have you 
earned? (Bachelor’s 
degree is listed) 

lb. What was your major 
field of study for each 
degree? 

lc. In what year did you 
receive each degree? 

No items about second 
major or minor field of 
study 


2a. Do you have a 
bachelor’s degree? 

2b. What was your major 
field of study? 

2c. In what year did you 
receive your bachelor’s 
degree? 

2d. Did you have a second 
major or minor field of 
study? 

2e. What was you second 
major or minor field of 
study? 


5a. Do you have a 
bachelor’s degree? 

5b. What was your major 
field of study? 5c. In what 
year did you receive your 
bachelor’s degree? 

5d. Did you have a second 
major field of study? 

5e. What was your second 
major field of study? 

5f. Did you have a minor 
field of study? 

5g. What was your minor 
field of study? 


Second BA/BS 


la. Which of the following 
college degrees have you 
earned? (Second 
Bachelor’s degree is 
listed) 

lb. What was your major 
field of study for each 
degree? 

lc. In what year did you 
receive each degree? 

No items about second 
major or minor field of 
study 


na 


7a. Do you have a second 
bachelor’s degree? 

7b. What was your major 
field of study? 7c. In what 
year did you receive your 
second bachelor’s degree? 


MA/MS 


la. Which of the following 
college degrees have you 
earned? (Master’s degree 
is listed) 

lb. What was your major 
field of study for each 
degree? 

lc. In what year did you 
receive each degree? 


3a. Do you have a master’s 
degree? 

3b. What was your major 
field of study? 

3c. In what year did you 
receive your master’s 
degree? 


8a. Do you have a master’s 
degree? 

8b. What was your major 
field of study? 8c. In what 
year did you receive your 
master’s degree? 
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Item topic 


1987-88 


1990-91 


1993-94 


Second MA/MS 


la. Which of the following 
college degrees have you 
earned? (Second Master’s 
degree is listed) 

lb. What was your major 
field of study for each 
degree? 

lc. In what year did you 
receive each degree? 


na 


9a. Do you have a second 
master’s degree? 

9b. What was your major 
field of study? 

9c. In what year did you 
receive your second 
master’s degree? 


Other degrees 


Similar series for: 

Associate degree or 
vocational certificate 
Professional diploma or 
education specialist (At 
least one year beyond M.A. 
level) 

Doctorate (e.g., Ph.D.., 
Ed. D.) 

First Professional degree 
(e.g., M.D., L.L.B., J.D., 
D.D.S.) 

No Degree or diploma 


Similar series for: 

Associate degree 

Education specialist or 
professional diploma (at 
least one year beyond 
Master’s level) 

Doctorate or first 
professional degree (Ph.D., 
Ed.D., M.D., L.L.B., J.D., 
D.D.S.) 


Similar series for: 

Associate degree 

Education specialist or 
professional diploma (at 
least one year beyond 
master’s level) 

Doctorate or first 
professional degree (Ph.D., 
Ed.D., M.D., L.L.B., J.D., 
D.D.S.) 


Teaching 

experience 


2a. How many years of 
elementary or secondary 
teaching experience did you 
have prior to becoming a 
principal? If less than one 
year, enter “1". 


5a. How many years of 
elementary or secondary 
teaching experience did 
you have PRIOR to 
becoming a principal? 
Count part of a year as 1 
year. 


1 la. How many years of 
elementary or secondary 
teaching experience did 
you have PRIOR to 
becoming a principal? 
Count part of a year as 1 
year. 




2b. How many years of 
elementary or secondary 
teaching experience have 
you had altogether? 


5b. How many years of 
elementary or secondary 
teaching experience have 
you had SINCE becoming 
a principal? Count part of 
a year as 1 year. 


lib. How many years of 
elementary or secondary 
teaching experience have 
you had SINCE becoming 
a principal? 

Count part of a year as 1 
year. 
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Item topic 


1987-88 


1990-91 


1993-94 


Main assignment 
in last year of 
teaching 


3a. In your last year of 
teaching before you became 
an administrator, what was 
the filed of your 
PRIMARY TEACHING 
ASSIGNMENT, i.e., the 
filed in which you taught 
the most classes? If your 
teaching schedule was 
divided equally between 
two fields, record either 
field as your primary 
assignment field, mark box 
1, and enter the second 
field in item 3b. 


6. In your most recent year 
of teaching, what was the 
field of your MAIN 
TEACHING 
ASSIGNMENT, i.e., the 
filed in which you taught 
the most classes? If your 
teaching schedule was 
divided equally between 
two fields, record either 
field as your main 
assignment field. 


12. In your most recent 
year of teaching, what was 
the field of your MAIN 
TEACHING 
ASSIGNMENT, i.e., the 
filed in which you taught 
the most classes? If your 
teaching schedule was 
divided equally between 
two fields, record either 
field as your main 
assignment field. 


Other school 
positions 


4. What other positions, if 
any, did you hold before 
you became a principal? 
Checklist for other 
positions — nothing about 
years. 


7. What other school 
positions, if any, did you 
hold before you became a 
principal? Checklist for 
other positions — 
nothing about years. 


14a. Did you hold any 
other school position 
BEFORE you became a 
principal? Which of the 
following school positions 
did you hold before 
becoming a principal and 
for how many years? 




1) Department head or 
curriculum coordinator 


1 ) Department head or 
curriculum coordinator 


1) Department head 

2) Curriculum specialist 
or coordinator 




2) Assistant principal or 
program director 


2) Assistant principal or 
program director 


3) Assistant principal or 
program director 




3) Guidance counselor 


3) Guidance counselor 


4)Guidance counselor 




na 


na 


5) Library media 
specialists/librarians 




4) Athletic coach 


4) Athletic coach 


6) Athletic coach 




5) Sponsor for student 
clubs, debate teams 


5) Sponsor for student 
clubs, debate teams 


7) Sponsor for student 
clubs, debate teams 




na 


6) Other - SPECIFY 


8) Other - Describe other 
school position 
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Item topic 


1987-88 


1990-91 


1993-94 




6) None of the above 


0) None 


na 


Aspiring 

administrator 

training 


na 


8a. Prior to becoming an 
administrator, did you 
participate in any district 
or school training or 
development program for 
ASPIRING school 
administrator 


15a. Prior to becoming a 
principal, did you 
participate in any district 
or school training or 
development program for 
ASPIRING school 
administrator 


Indian 

administrator 

training 


na 


8b. Have you ever 
participated in a training 
program for Indian 
education administrators? 


15b. Have you ever 
participated in a training 
program for Indian 
education administrators? 


Principal 

training 


6. Aside from college 
coursework for a degree, 
have you had any of the 
following types of training 
for your current position? 


10. Aside from college 
coursework for a degree, 
have you had any of these 
types of training for your 
current position? 


20. Aside from college 
coursework for a degree or 
participation in a 
program for ASPIRING 
school principals, have 
you had any of these types 
of training for your current 
position? 




1) In-service training in 
evaluation and supervision 


1) In-service training in 
evaluation and supervision 


1) In-service training in 
evaluation and supervision 




2) Training in management 
techniques 


2) Training in management 
techniques 


2) Training in management 
techniques 




3) An administrative 
internship 


3) An administrative 
internship 


3) An administrative 
internship 




4) None of the above 


4) None of the above 


4) None of the above 


other positions 


5. Prior to this school year, 
and since earning your 
first college degree, how 
many years have you been 
employed in each of the 
following position? If less 
than one year, enter “1.” 


9. Prior to this school year, 
how many years have you 
been employed in each of 
the following position? If 

less than one year, enter 
“1.” 


17. Prior to this school 
year, how many years have 
you been employed in each 
of the following position? 
Count part of a year as 1 
year. 




a) As the principal in this 
school? 


a) As the principal in this 
school? 


a) As the principal in this 
school? 
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Item topic 


1987-88 


1990-91 


1993-94 




b) As the principal in other 
schools? 


b) As the principal in other 
schools? 


b) As the principal in other 
schools? 




c) In other school or district 
administrative positions? 


c) In other school or 
district administrative 
positions? 


na 




d) In other nonteaching 
nonadministrative positions 
in elementary and secondary 
education? 


d) In other nonteaching 
nonadministrative 
positions in elementary 
and secondary education, 
e.g., a guidance counselor 
or school psychologist? 


na 




e) In positions outside 
elementary and secondary 
education? 


e) In professional 
positions outside 
elementary and secondary 
education? 


na 


Planned tenure 
as a principal 


na 


11a. How long do you plan 
to remain a principal? 


21b. How long do you plan 
to remain a principal? 




na 


1 lb. In how many years 
do you plan to retire from 
your position as a 
principal? 


21c. In what year do you 
PLAN to retire from your 
position as a principal? 


Salary 


,7a. What is your annual 
salary from this school this 
year before taxes and 
deductions? 


12a. What is your current 
ANNUAL salary for this 
position before taxes and 
deductions? 


22a. What is your current 
ANNUAL salary for this 
position before taxes and 
deductions? 




7b. For how many months 
of the year are you 
employed as the 
administrator in this 
school? 


1 2b. For how many months 
of the year are you 
employed as the 
administrator in this 
school? 


22b. For how many months 
of the year are you 
employed as the principal 
in this school? 


Benefits 


8. Which of these benefits 
do you receive, in whole or 
in part, from this school (or 
district) in addition to your 
salary? 


13. Which of these benefits 
do you receive, in whole or 
in part, from this school or 
district in addition to your 
salary? 


23. Which of these benefits 
do you receive, in whole or 
in part, from this school or 
district in addition to your 
salary? 
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Item topic 


1987-88 


1990-91 


1993-94 




1 ) Housing or housing 
expenses 

2) Meals 

3) Tuition for your 
children 

4) College tuition for 
yourself 

5) General medical 
insurance 

6) Dental insurance 

7) Group life insurance 

8) Car/transportation 
expenses 

9) Pension contributions 

na 

0) None of the above 


1) Housing or housing 
expenses 

2) Meals 

3) Tuition for your 
children 

4) College tuition for 
yourself 

5) General medical 
insurance 

6) Dental insurance 

7) Group life insurance 

8) Car/transportation 
expenses 

9) Pension contributions 

na 

0) None of the above 


5) Housing or housing 
expenses 

6) Meals (including free 
or reduced-price lunch) 

na 

8) Reimbursement for 
tuition and course fees 

1) General medical 
insurance 

2) Dental insurance 

3) Group life insurance 

7) Car/transportation 
expenses 

4) Pension contributions 

9) Child care 

0) None of the above 




14. For each of the 
following matters, indicate 
whether it is a serious 
problem, a moderate 
problem, a minor problem, 
or not a problem in this 
school. 


14. For each of the 
following matters, 

indicate whether it is a 
serious problem, a 
moderate problem, a minor 
problem, or not a problem 
in this school. 


24. To what extent is 
each of the following 
matters a problem in this 
school? Indicate whether 
it is a serious problem, a 
moderate problem, a minor 
problem, or not a problem 
in this school 


school 

environment 


b. Student absenteeism 


c. Student absenteeism 


b. Student absenteeism 




a. Student tardiness 


b. Student tardiness 


a. Student tardiness 




c. Teacher absenteeism 


a. Teacher absenteeism 


c. Teacher absenteeism 




d. Students cutting class 


d. Students cutting class 


d. Students cutting class 




e. Physical conflicts among 
students 


g. Physical conflicts 
among students 


e. Physical conflicts among 
students 




f. Robbery or theft 


h. Robbery or theft 


f. Robbery or theft 
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Item topic 


1987-88 


1990-91 


1993-94 




g. Vandalism of school 
property 


i. Vandalism of school 
property 


g. Vandalism of school 
property 




h. Student pregnancy 


j. Student pregnancy 


h. Student pregnancy 




i. Student use of alcohol 


k. Student use of alcohol 


i. Student use of alcohol 




j. Student drug use 


1. Student drug abuse 


j. Student drug abuse 




k. Student possession of 
weapons 


m. Student possession of 
weapons 


k. Student possession of 
weapons 




m. Verbal abuse of teachers 


p. Verbal abuse of teachers 


1. Verbal abuse of teachers 




na 


n. student disrespect for 
teachers 


m. Student disrespect for 
teachers 




na 


e. Students dropping out 


n. Students dropping out 




na 


f. Student apathy 


o. student apathy 




na 


q. Lack of academic 
challenge 


p. Lack of academic 
challenge 




na 


r. Lack of parent 
involvement 


q. Lack of parent 
involvement 




na 


s. Parental alcoholism 
and/or drug abuse 


r. Parental alcoholism 
and/or drug abuse 




na 


t. Poverty 


s. Poverty 




na 


u. Racial tension 


t. Racial tension 4 




na 


na 


u. Students come to 
school unprepared to 
learn 




na 


na 


v. Poor nutrition 




na 


na 


w. Poor student health 




na 


na 


x. Student problems with 
the English language 




1. Physical abuse of teachers 


o. Physical abuse of 
teachers 


na 
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Item topic 


1987-88 


1990-91 


1993-94 




na 


v. Cultural conflict 


na 


Decision making 
influence: Stem 


16. Using the scale 1-6, 
indicate how much 
ACTUAL influence you 
think each group or person 
has on decisions concerning 
the following activities : 


15. Using the scale 1-6, 
where 0 is ’none’ and 5 is 
’a great deal,’ indicate 
how much actual influence 
you think each group or 
person has on decisions 
concerning the following 
activities: 


25. Using the scale 0-5, 
where 0 is ’none 9 and 5 is 
’a great deal/ indicate 
how much ACTUAL 
influence you think each 
group or person has on 
decisions concerning the 
following activities: 


Establishing 

curriculum 


na 

na 

1) School 

district/Governing Board 

2) Principal/head 

3) Teachers 
na 

na 

na 


1) State Department of 
Education 

na 

2) School board 

3) Principal 

4) Teachers 

na 

5) Librarians/media 
specialists 

6) Parent association 


1) State Department of 
Education 

2) School district staff 

3) School board 

4) Principal 

5) Teachers 

6) Curriculum specialists 

7) Library media 
Specialists/Librarians 

8) Parent association 


Hiring new full- 
time teachers 


na 

na 

1) School 

district/Governing Board 

2) Principal/head 

3) Teachers 
na 


na 

na 

1) School board 

2) Principal 

3) Teachers 

4) Parent association 


1) State Department of 
Education 

2) School district staff 

3) School board 

4) Principal 

5) Teachers 

6) Parent association 


Setting 

discipline policy 


na 

na 

1) School 

district/Governing Board 

2) Principal/head 

3) Teachers 
na 


1) State Department of 
Education 

na 

2) School board 

3) Principal 

4) Teachers 

5) Parent association 


1 ) State Department of 
Education 

2) School district staff 

3) School board 

4) Principal 

5) Teachers 

6) Parent association 
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Item topic 


1987-88 


1990-91 


1993-94 




na 


na 


Deciding how the school 








budget will be spent 




na 


na 


Determining content of 








in-service programs 




na 


na 


Evaluating teachers 


Educational 


na 


16. We are interested in 


26. We are interested in 


goals: stem 




the importance you place 


the importance you place 






on various educational 


on various educational 






goals. From the following 


goals. From the following 






eight goals, which do you 


eight goals, which do you 






consider the most 


consider the most 






important, the second most 


important, the second most 






important, and the third 


important, and the third 






most important? 


most important? 


Educational 


na 


1) Building basic literacy 


1) Building basic literacy 


goals list 




skills (reading, math, 


skills (reading, math, 






writing, speaking) 


writing, speaking) 






2) Encouraging academic 


2) Encouraging academic 






excellence 


excellence 






3) Promoting occupational 


3) Promoting occupational 






or vocational skills 


or vocational skills 






4) Promoting good work 


4) Promoting good work 






habits and self-discipline 


habits and self-discipline 






5) Promoting personal 


5) Promoting personal 






growth (self-esteem, self- 


growth (self-esteem, self- 






knowledge, etc.) 


knowledge, etc.) 






6) Promoting human 


6) Promoting human 






relations skills 


relations skills 






7) Promoting specific 


7) Promoting specific 






moral values 


moral values 






8) Promoting multicultural 


8) Promoting multicultural 






awareness or 


awareness or 






understanding 


understanding 


Gender 


9. Are you male or female? 


18. Are you male or 


27. Are you male or 






female? 


female? 
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Item topic 


1987-88 


1990-91 


1993-94 


Race 


10. What is your race? 

na 


19a. What is your race? 
19b. Axe you enrolled in a 
state or federally 
recognized tribe? 


28a. What is your race? 

28b. Are you enrolled in a 
state or federally 
recognized tribe? 


Hispanic origin 


1 1 . Are you of Hispanic 
origin? 


20. Are you of Hispanic 
origin? 


29. Are you of Hispanic 
origin? 


Age 


12. What is your year of 
birth? 


21 . What is your year of 
birth? 


30. What is your year of 
birth? 
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Appendix B. 



Comparison of Items on the SASS School 
Questionnaire 



5K 



A feasibility Study of Longitudinal Design for Schools and Staffing Survey 



Data items comparable between 1987-88, 1990-91, and 1993-94 SASS 
School Questionnaire (Differences shown in bold, na=not asked) 



Item topic 


1987-88 


1990-91 


1993-94 


Enrollment 


1. How many students (in 
head counts) were 
enrolled in grades K- 12 
in this school on or 
about October 1, 1987? 


1. How many students (in 
head counts) were 
enrolled in this school in 
grades K-12 or 
comparable ungraded 
levels? Include only 
students enrolled in the 
school named on the 
questionnaire label. Do 
NOT include 
prekindergarten or 
postsecondary students. 


8. What was the total 
number of students 
enrolled in this school 
around the first of 
October? (Summing after 
listing all grades, K- 12 and 
ungraded) 


Minority 

enrollment 


9. How many students 
attending this school are 

— a. American Indian or 
Alaskan Native 


9. How many K- 12 
students in this school are 
(Do NOT include 
prekindergarten or 
postsecondary students): 
a. American Indian or ... 


9. Around the first of 
October, how many 
students were: Do not 
include prekindergarten, 
postsecondary, or adult 
education students, and 
children who are enrolled 
only in day care at this 
school. a.American Indian 
or... 


Male 

enrollment 


4b. What percentage of 
students enrolled in this 
school are male? 


4. What percent of K- 12 
students enrolled in this 
school are males? Do NOT 

include prekindergarten 
or postsecondary students. 


10. How many MALE 
students attended this school 

around the first of 
October? 


Requirements 
for admission 


15. Which of the 
following does this school 
use for admission? (Nine 
options, with a “None of 
these (This school has no 
special requirements for 
admission) 


16a. Does this school have 
any special requirements for 
admission other than proof 
of immunization, age, or 
residence? 


13a. Does this school have 
any special requirement for 
admission other than proof of 
immunization, age, or 
residence? (a list of 
requirement, the most 
important, no the second 
most important) 
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Item topic 


1987-88 


1990-91 


1993-94 


Specific 

requirements 

(list) 


(1) Admission test, (2) 
Standardized achievement 
test, (3) Academic record, 

(4) Special student needs, 

(5) Special student 
aptitudes, (6) Personal 
interview, (7) 
Recommendations, (8) 
Something else 
(SPECIFY), (9) None of 
these (This school has no 
special requirements for 
admission) 


(1) Admission test, (2) 
Standardized achievement 
test, (3) Academic record, 

(4) Special student needs, 

(5) Special student 
aptitudes, (6) Personal 
interview, (7) 
Recommendations 


(1) Admission test, (2) 
Standardized achievement 
test, (3) Academic record, (4) 
Special student needs, (5) 
Special student aptitudes, (6) 
Personal interview, (7) 
Recommendations, (8) None 
of the above 


Most important 
criteria 


na 


16c. Of the categories 
marked for item 16b above, 
which is the most important 
consideration for admission 

and which is the second 
most important? 


14c. Of the categories you 
marked for question 13b 
above, which is the most 
important consideration for 
admission? 
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6(7 



A Feasibility Study oj Longitudinal Design for Schools and Staffing Survey 



Item topic 


1987-88 


1990-91 


1993-94 


Length of 
school day 


na 


7. How long is the school 
day for most students in this 
school? If the length of day 
varies by grade level, record 
the longest day. 


12. How long is the school 
day for students in this 
school? If the length of day 
varies by grade level, record 
the longest day. 
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Item topic 


1987-88 


1990-91 


1993-94 


Type of school 


3. Which of the following 
best describes this 
school? (1) REGULAR 
elementary or secondary, 
(2) Elementary or 
secondary with a special 
program emphasis (e.g., 
science/math magnet 
school, performing arts 
high school, gifted/ 
talented school 
(SPECIFY), (3) SPECIAL 
EDUCATION- serves 
primarily handicapped 
students, (5) 

VOCATIONAL/TECHNI 
CAL (serves primarily 
students being trained for 
occupations,) (4) 
ALTERNATIVE (Offers a 
curriculum designed to 
address the needs of 
students which typically 
cannot be met in a 
regular school; provides 
nontraditional education; 
may be an adjunct to a 
regular school. Does not 
specifically fall into 
regular, special 
education or vocational 
education school 
categories. (SPECIFY) 


3. What type of school 
this?(l) REGULAR 
elementary or secondary, 

(2) Elementary or secondary 
with a SPECIAL 
PROGRAM EMPHASIS 
e.g., science/math school, 
performing arts high school, 
talented/gifted school, 
foreign language 
immersion school, etc., (3) 
SPECIAL EDUCATION- 
serves primarily 
handicapped students, (4) 
VOCATIONAL/TECHNIC 
AL - serves primarily 
students being trained for 
occupations, (5) 
ALTERNATIVE - offers a 
curriculum designed to 
provide alternative or 
nontraditional education; 
does not specifically fall 
into regular, special 
education, or vocational 
school 


14. What type of school this? 
(1) REGULAR elementary or 
secondary, (2) Elementary or 
secondary with a SPECIAL 
PROGRAM EMPHASIS - 
e.g., science/math school, 
performing arts high school, 
talented/gifted school, 
foreign language 
immersion school, etc., (3) 
SPECIAL EDUCATION- 
primarily serves students 
with disabilities, (4) 
VOCATIONAL/TECHNICA 
L - primarily serves students 
being trained for 
occupations, (5) 
ALTERNATIVE - offers a 
curriculum designed to 
provide alternative or 
nontraditional education; 
does not specifically fall into 
the categories of regular, 
special education, or 
vocational school 


# of students 
absent 


8. What percentage of 
students were not in 
attendance today? 
Include both excused and 
unexcused absences. 


8. How many K-12 
students were absent the 
most recent school day? 

Include both excused and 
unexcused absences. Do 
NOT include 
prekindergarten or 
postsecondary students. 


1 1 . How many students were 
ABSENT on the most recent 
school day? 
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Item topic 


1987-88 


1990-91 


1993-94 


Number of staff 
(Instructions) 


26. For each of the 
following categories, how 
many employees 
regularly worked in this 
school on or about 
October 1, 1987? 

(Report totals in full-time 
equivalents (FTE’s) to 
the nearest tenth.) 


31. How many employees 
hold full- or part-time 
positions in this school in 
each of the following 
categories? If an 
employee holds a position 
in more than one of the 
categories, count that 
person as part-time in 
each category that applies 


16. How many staff held 
PART-TIME position in 
this school in each of the 
following categories 
around the first of 
October? INCLUDE AS 
PART TIME: Employees 
who work part time, 
Employees you share with 
other schools within or 
outside of the school 
district. Employees who 
perform more than one 
function at this school; for 
example, a teaching 
principal would be counted 
once as a part-time teacher 
and again as a part-time 
principal. 

17. How many staff held 
FULL-TIME positions in 
this school in each of the 
following categories 
around the first of 
October? 




a. Principals and 
assistant principals 


a. Principal(s) 


a. Principals 




see above 


b. Assistant principal(s) 


b. Vice principals and 
assistant principals 




c. Guidance counselors 


c. Guidance counselors 


d. School counselors 




na 


d. Vocational counselor 


d. School counselors 




d. Librarians and other 
professional media staff 


e. Librarians and other 
professional media staff 


e. Library media 
specialists/ librarians 




na 


h. Library or media center 
aides 


h. Library media center aides 



BEST COPY AVAILABLE 



A Feasibility Study of Longitudinal Design for Schools and Staffing Survey 



63 



Item topic 


1987-88 


1990-91 


1993-94 




e. Other professional 
staff such as curriculum 
specialists, 
administrative and 
business staff, and social 
workers 


f. Other professional staff 
such as curriculum 
specialists, administrative 
and business staff, social 
workers, and health 
professionals 


c. Instructional 
coordinators & 
supervisors, such as 
curriculum specialists 




na 


na 


f. Student support services 
professional staff, such as 
school psychologists, social 
workers, occupational 
therapists, speech 
therapists, and nurses 




na 


na 


j. Secretaries and other 
clerical support staff 




na 


i. All other 

noninstructional staff 
(include maintenance, 
food service, and clerical 
staff) 


k. Other employees (e.g., 
cafeteria workers, 
maintenance staff, etc.) 




f. Teacher aides 

(paraprofessionals who 
assist teachers) 


g. Classroom teacher aides 
(paraprofessionals who 
assist classroom teachers) 


i. Teacher aides 




b. Teachers (different 
definition —exclude 
substitute) 


24a. How many K- 12 
teachers have FULL- 
TIME teaching positions 
at THIS school? 

24b. How many K- 12 
teachers have PART- 
TIME teaching positions 
at THIS school? Include 
itinerant teachers. 


g. Teachers (Long, detailed 
definitions and 
instructions. Includes some 
substitutes) 
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Item topic 


1987-88 


1990-91 


1993-94 


Minority 

teachers 


10. How many teachers 
in this school are — (race 
categories) 


25. How many K-12 
teachers in this school are 
— (race categories) 
Include both full- and 
part-time teachers 


1 8. Around the first of 
October, how many part- 
time and full-time 
TEACHERS in this school 
were: (race categories) Do 
not include teachers who 
teach ONLY 
prekindergarten, 
postsecondary, or adult 
education 


Teacher 

absence 


na 


26. How many K- 12 
teachers were absent the 
most recent school day? 
Include both full- and part- 
time teachers. 


19. How many part-time and 
full-time teachers were 
absent the most recent school 
day? 


Program filter 
item 


1 1 . For each of the 
following programs or 
services, please indicate 
whether it is available to 
students in this school, 
either during or outside of 
regular school hours, and 
regardless of funding 
sources. 

If you mark “Yes” for 
a program or service, 
record the number of 
students served. 


10. For each of the 
following programs or 
services, please indicate 
whether it is available to 
students in this school, 
either during or outside of 
regular school hours, and 
regardless of funding 
sources. Do not include 
prekindergarten or 
postsecondary students. 

IF YES: How many 
students participate in this 
program/received this 
service? 


22. Please indicate whether 
each of the following 
programs or services is 
currently available at this 
school either during or 
outside of regular school 
hours, and regardless of 
funding. Do not include 
programs available only to 
prekindergarten students. 
Include only those who are 
enrolled in this school. Do 
not include 
prekindergarten, 
postsecondary, or adult 
education students, and 
children who are enrolled 
only in day care at this 
school. 

IF YES: How many 
students participate in this 
program? 
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Item topic 


1987-88 


1990-91 


1993-94 


Programs & 
services offered 

# of students 
participated 


c. Remedial reading - 
organized compensatory, 
diagnostic, and remedial 
activities designed to 
correct and prevent 
difficulties in the 
development of reading 
skills. 


c. Remedial reading - 
organized compensatory, 
diagnostic, and remedial 
activities designed to 
correct and prevent 
difficulties in the 
development of reading 
skills. 


a. Remedial reading - 
organized compensatory, 
diagnostic, and remedial 
activities designed to correct 
and prevent difficulties in the 
development of reading 
skills. Includes remedial 
reading instruction that is 
part of special education 
and Chapter 1 programs, 
as well as other remedial 
reading programs. 




d. Remedial mathematics - 
organized compensatory, 
diagnostic, and remedial 
activities designed to 
correct and prevent 
difficulties in the 
development of 
mathematics skills. 


d. Remedial mathematics - 
organized compensatory, 
diagnostic, and remedial 
activities designed to 
correct and prevent 
difficulties in the 
development of 
mathematics skills. 


b. Remedial mathematics - 
organized compensatory, 
diagnostic, and remedial 
activities designed to correct 
and prevent difficulties in the 
development of mathematics 
skills. Includes remedial 
math instruction that is 
part of special education 
and Chapter 1 programs, 
as well as other remedial 
math programs. 




e. Programs for the 
handicapped - Instruction 
for the mentally retarded, 
specific learning disabled, 
physically handicapped, 
and other handicapped. 


e. Programs for 
handicapped students 
- instruction for the 
mentally retarded, specific 
learning disabled, physically 
handicapped, and other 
handicapped. 


c. Programs for students 
with disabilities - 
Instruction for the mentally 
retarded, specific learning 
disabled, physically 
disabled, and other students 
with disabilities. 




f. Programs for the gifted 
and talented - activities 
designed to permit gifted 
and talented students to 
further develop their 
abilities. 


f. Programs for the gifted 
and talented - activities 
designed to permit gifted 
and talented students to 
further develop their 
abilities. 


d. Programs for the gifted 
and talented - Activities 
designed to permit gifted and 
talented students to further 
develop their abilities. 




i. Extended day or before- 
or after-school day-care 
programs 


h. Extended day or before- 
or after-school day-care 
programs 


e. Extended day or before- or 
after-school day-care 
programs 
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Item topic 


1987-88 


1990-91 


1993-94 




b. English as a Second 
Language - Students with 
limited English proficiency 
are provided with intensive 
instruction in English. 


a. English as a Second 
Language - Students with 
limited English proficiency 
are provided with intensive 
instruction in English. 


f. English as a Second 
Language - Students with 
limited English proficiency 
are provided with intensive 
instruction in English. 




a. Bilingual education - 
Native language is used to 
varying degrees in 
instructing students with 
limited English 
proficiency. (Includes, 
for example, transitional 
bilingual education and 
structured immersion.) 


b. Bilingual education - 
native language is used to 
varying degrees in 
instructing students with 
limited English proficiency. 
For example, transitional 
bilingual education and 
structured immersion. 


g. Bilingual education - 
Native language is used to 
varying degrees in instructing 
students with limited English 
proficiency. For example, 
transitional bilingual 
education and structured 
immersion. Do not include 
foreign language classes or 
foreign language 
immersion programs. 




h. Diagnostic & 
prescriptive services - 
services provided by 
trained professionals to 
diagnose learning 
problems of students and 
to plan and provide 
therapeutic or educational 
programs based upon such 
services. 


g. Diagnostic & prescriptive 
services - services provided 
by trained professionals to 
diagnose learning problems 
of students and to plan and 
provide therapeutic or 
educational programs based 
upon such services. 


h. Diagnostic & prescriptive 
services - Services provided 
by trained professionals to 
diagnose learning problems 
of students and to plan and 
provide therapeutic or 
educational programs based 
upon such services. 




na 


na 


i. Medical health care 
services - Services 
provided by trained 
professionals (e.g., 
physician, physician 
assistant, nurse, or nurse 
practitioner) to diagnose 
and treat health problems 
of students. 


Library 


na 


13. Does this school have a 
library/media center? 


23. Does this school have a 
media center/library? 
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Item topic 


1987-88 


1990-91 


1993-94 


Kindergarten 

program 


16a. For what grade 
levels does your school 
offer instruction? (KG 
option) 


12a. Does this school offer 
a KINDERGARTEN 
program? 


25a. Around the first of 
October, did this school 
offer a kindergarten 
program? 


length of school 
day (KG) 


na 


12c. How long is the school 

day for the kindergarten 
program? 


25b. How long is the school 

day for a kindergarten 
student? 


length of school 
week (KG) 


na 


12b. How many days per 
week do the kindergarten 
students attend this school? 


25c. How many times per 
week does a kindergarten 
student attend? If the 
number of days per week 
varies (e.g., some students 
attend 3 days per week and 
some attend 5 days per 
week), record the most 
days that a student would 
attend in a week. 


Chapter 1 


12a. Does this school 
provide ECIA Chapter 1 
services? 


14a. Does this school 
provide Chapter 1 
services under the 
Elementary and 
Secondary Education Act 
as amended, i.e., federal 
funds for the special 
educational needs of 
disadvantaged children? 


27a. Around the first of 
October, did any students 
enrolled in this school 
receive Chapter 1 services 
at this school, or any other 
location? Chapter 1 is a 
federally-funded program 
which provides educational 
services, such as remedial 
reading or remedial math, 
to children who live in 
areas with high 
concentrations of low- 
income families. 


Chapter 1 
enrollment 


12b. How many students 
are served? 


14b. How many students 
are served 


27b. How many students 

enrolled in this school 
received Chapter 1 services 
at this school, or any other 
location? Reporta 
separate count for 
prekindergarten-age 
children. 
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Item topic 


1987-88 


1990-91 


1993-94 


Chapter 1 
staffing 


na 


13c. How many Chapter 1 
teachers and/or teacher 
aides are at this school? 


27c. In head counts, how 
many Chapter 1 teachers and 
teacher aides were teaching 
at this school around the first 
of October? 


Free/reduced 
price lunch 
Program 
eligibility 


13a. Are any of the 
students in this school 
eligible for free or 
reduced- price lunches that 
are paid for with public 
funds, e.g., Federal 
government or other 
government? 


15a. Are any of the students 
in this school eligible for 
free or reduced-price 
lunches that are paid for 
with public funds, e.g.. 
Federal government or other 
government? (yes, no) 


28b. Regardless of whether 
this school participates in 
this National School Lunch 
Program, were any 
students in this school 
ELIGIBLE for the 
program? (yes, no, don’t 
know) 


Program 

participation 


Inferable from next item 


Inferable from next item 


28a. Does this school 
participate in the National 
School Lunch Program? 


Number of 

students 

participating 


13c. How many students 
receive free or reduced 
price lunches? 


15b. How many students 
receive free or reduced- 
price lunches? 


28d. Around the first of 
October, how many 
students at this school 
received free or reduced- 
price lunches through the 
National School Lunch 
Program? This number 
may differ from the 
number of applicants 
approved, depending upon 
how the program is 
implemented. Report a 
separate count for 
prekindergarten-age 
students. 


Number of 

students 

approved 


na 


na 


28c. Around the first of 
October, how many 
applicants at this school 
were approved for the 
National School Lunch 
Program? 
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Item topic 


1987-88 


1990-91 


1993-94 


Voc./tech. 

program 


28f. (PART OF 
PROGRAM/ SERVICES 
ITEM) Vocational or 
technical programs - 
instruction designed to 
provide students with 
occupational skills 
needed for work 


21a. Does this school offer 
a vocational/technical 
program? 


29c. Does this school have a 
M Tech-Prep M program, i.e., 
vocational/ technical 
instruction in the last two 
years of high school 
designed to prepare 
students for two years of 
vocational instruction at 
the postsecondary level? 
NOTE: This item follows a 
filter question asking if 
there are 12th grade 
students. 


Grade 12 
presence 


16a. For what grade 
levels does your school 
offer instruction (Grade 
12 listed) 


19. Does this school (the 
school named on the 
questionnaire label) 
provide instruction for 
grade 12? 


29a. Does this school 
provide instruction to 
students in grade 12? 


Grade 12 
enrollment: 
previous year 


19. How many students 
were enrolled in 12th 
grade on or about 
October 1, 1986 (last 
year)? 


23a. LAST SCHOOL 
YEAR, how many students 
were enrolled in 12th 
grade? 


30a. Last school year, were 
any students enrolled in 
12th grade? How many 
students? 


Graduation rate 


20. How many students 
were graduated from 1 2th 
grade last year? Include 
1987 summer graduates. 


23b. How many students 
were graduated from the 
12th grade last year? 
Include 1990 summer 
graduates. 


30c. How many students 
were graduated from the 1 2th 
grade last year? Include 
1993 summer graduates. Do 

not include students who 
received only vocational 
certificates, certificates of 
attendance, or certificates 
of completion. 


College 

application 


21. What is the estimated 
percentage of last year’s 
graduates that applied to 
a two- or four-year 
college? 


23c. How many of last 
year’s graduates applied to 
two- or four-year colleges? 


30d. How many of last year’s 
graduates applied to two- or 
four-year colleges? 
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Item topic 


1987-88 


1990-91 


1993-94 


Teaching 

vacancies 


na 


33a. Were there teaching 
vacancies in this school for 
this school year, i.e., 
teaching positions for which 
teachers were recruited and 
interviewed? 


20a. Were there teaching 
vacancies in this school for 
this school year, i.e., teaching 
positions for which teachers 
were recruited and 
interviewed? 


Methods used 
to fill vacancies 


na 


33c. Which of these 
methods did this school use 
to cover the vacancy(ies)? 


20b. Which of these methods 
did this school use to cover 
the vacancy(ies)? 




na 


na 


1) Hired a fully qualified 
teacher 




na 


7) Hired a less qualified 
teacher 


2) Hired less than fully 
qualified teacher 




na 


1) Cancelled planned course 
offerings 


3) Canceled planned course 
offerings 




na 


2) Expanded some class 
sizes 


4) Expanded some class sizes 




na 


3) Added sections to other 
teachers’ normal teaching 
loads 


5) Added sections to other 
teachers’ normal teaching 
loads 




na 


4) Assigned a teacher of 
another subject or grade 
level to teach those classes 


6) Assigned a teacher of 
another subject or grade level 
to teach those classes 




na 


na 


7) Assigned an 
administrator or counselor 
to teach the class 




na 


5) Used long-term and/or 
short-term substitutes 


8) Used long-term or short- 
term substitutes 




na 


6) Used part-time or 
itinerant teachers 


na 
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Item topic 


1987-88 


1990-91 


1993-94 


Difficulty in 
hiring different 
types of 
teachers 


na 


33d. How difficult or easy 
was it to fill the vacancies 
for this school year in each 
of the following fields: (1) 
General elementary, (2) 
special education, (3) 
English, (4) Mathematics, 
(5) Physical sciences, (6) 
Biology or life sciences, (7) 
English as a second 
language (ESL) or 
bilingual education, (8) 
Foreign language, (9) 
Vocational education 


20c. How difficult or easy 
was it to fill the vacancies for 
this school year in each of 
the following fields: 

Includes a “Not applicable 
in this school option” (1) 
general elementary, (2) 
special education, 

(3)English, (4) Mathematics, 
(5) Physical sciences, (6) 
Biology or life sciences, 
(7)English as a Second 
Language (ESL), English 
for Speakers of Other 
Languages (ESOL), or 
bilingual education, (8) 
Foreign languages, (9) 

Music, (10) Business or 
Marketing, (11) Industrial 
arts, (12) Home economics, 

(13) Trade and industry, 

(14) Agriculture 



i‘\0 c 
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Appendix C. 



Memo: Thoughts on a Longitudinal Survey 
of Schools and Educational Reform 
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TO: 


Dan Kasprzyk, NCES 




Mary Rollefson, NCES 


FROM: 


David Baker, ESSI 


Date: 


1/13/97 


RE: 


Thoughts on a Longitudinal Survey of Schools and 




Educational Reform 



As per your request, I have collected some of my thoughts on the relative costs and benefits of the use of a 
longitudinal design to obtain information on schools and educational reform in the next SASS survey. Many 
of these ideas originate from the in-progress ESSI feasibility study that analyzes the “overlap 1990-91 and 
1993-94 SASS sample,” (i.e. Baker forthcoming). Before I present my thoughts and recommendations on 
these issues, I will briefly describe the capability of the current, cross-sectional SASS to study schooling over 
time. 



SASS has used a newly drawn sample of districts and schools for each of its three data collections; 
or, in other words, SASS is three separate cross-sectional surveys collected three years apart. But, although 
successive surveys have added new items, many of the same items have been collected at each wave. 
Therefore, unlike a one-time survey, SASS is a repeated, cross-sectional survey that can provide national and 
state estimates of variables at different points in time (i.e. simple trends). But because different schools are 
surveyed, the SASS cannot provide information on the dynamics of educational change at the school and 
district levels. Even though this is true for the full SASS public school sample, there is the capability in 
SASS to provide. some information on a subset of schools over time. 

To reduce sampling error, the 1990-91 and 1993-94 SASSs purposefully sampled a number (about 
30%) of districts and schools that had been surveyed in the previous SASS wave. Although this so-called 
“overlap sample” has never been used for substantive reporting, data from responses to the school 
questionnaire provide a view of what a longitudinal, panel (i.e. same schools over time) study might yield on 
the SASS items developed for a cross-sectional survey. (See the attached description of the feasibility study 
of the overlap sample.) 

I. What questions could a longitudinal survey of K- 12 th grade schools 
answer about reform? 

Regardless of whether one or many reforms are of interest, there are at least two basic questions about reform 
that longitudinal information on districts or schools could help to answer: 

1 . What is the pattern of implementation of reformed school policies and practices across place and 
time? 

2. What school outcomes, if any, are attributable to reforms? 

In the case of the first question, longitudinal information would enhance the quality of the answer that a 
cross-sectional survey could provide. For examples, while the cross-section SASS could provide information 
on the numbers of schools which use school-based management procedures, a longitudinal study would 
provide detailed information on which schools changed to more (or less) school-based management over a 
specific time period. But in the case of the second question, some type of a longitudinal design would be 
essential to providing a valid answer. Without pre-reform measures of outcomes such as achievement, 
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teacher satisfaction, drop-out rates, and so forth for schools, attributing effects to reformed practices becomes 
questionable. 

n. What are the relative costs and benefits of a longitudinal design for 
information on education reform in SASS? 

Benefits: Without a doubt there is an initial attractiveness of longitudinal study of reform. The notion of 
collecting useful information on the adoption and impact of educational reform of American schools is 
easily tied to considering some sort of a longitudinal survey design. Even the simplest of over time designs, 
such as following a penal across just two times points, has a clear appeal and usefulness in assessing the 
impact of new practices. Reforms are really changes in operating policy and practice, and thus they can be 
thought of as quasi-experimental treatments whose impact can be assessed by comparing a school’s 
functioning before and after implementation (question 2 above). And of course, there are more elaborate 
longitudinal designs available to compare reformed schools with unreformed schools over time. 

Besides impact of reform, longitudinal designs also come to mind when considering ways to gather 
information on impetus for reforms and their implementation across districts and schools (question 1 
above). The political environments of districts and schools are complex and the impetus for reform comes 
from a variety of sources without much, if any, centralized policy-setting in the nation. Therefore, although 
the national reform message may be more or less clear, reform is locally adopted and implemented. Within 
some broad common understanding of any single reform, corresponding policy and practice changes will 
rarely, if ever, be uniform across the numerous LEA’s and schools. Some districts and schools will adopt 
reform while others will not, and the timing of adoption will vary as well. Also, among those that choose to 
adopt reform, actual implementation can take different forms. Thus, although national education politics 
often produce pressure for educational change, the results of this pressure, falling upon a localized system, 
can yield substantial variation in reformed practice from location to location over time. Examining 
implementation over time across a panel of districts and/or schools is an attractive way to gather useful 
information on how reforms spread across the nation. 

This kind of reasoning has motivated recommendations to NCES for longitudinal approaches to other parts 
of its statistical portfolio (e.g. Singer and Willett 1996). The argument is that full-blown, multiple-waved 
longitudinal designs provide more information, and dynamically richer information, than simple cross- 
sections. 

This is true. Compared to a repeated, cross-sectional survey, a longitudinal panel of schools should provide 
a richer array of information. Even though SASS yields some basic information on trends over time and 
this could be exploited more effectively than it currently is, what is missing is more complex information on 
the dynamics of school change. Information such as which schools implemented a reform, how often 
during the time spell did they change practices, and what were the antecedents of adopting a reform is only 
provided through panel designs. Also the whole question of outcomes, as described above, is answered 
more rigorously with panel data. Another repeated cross-section of SASS, even without any longitudinal 
component, could provide some important reform information on adoption of reforms. But without some 
type of longitudinal design, the richer, and perhaps the more valuable, information on change related to 
reform will not be provided. 

Thus, in the abstract, longitudinal designs are hard to reject. It is akin to trying to argue in principle against 
larger and more stratified samples. Given the obvious attraction of longitudinal designs for collecting 
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information on educational reform, are there any drawbacks to recommending such a design for the next 
SASS? 

Costs: One problem is the cost of redesigning items and of increased response burden for a longitudinal 
design as applied to specific issues of reform in SASS. SASS is not a new survey and there are many items 
which need to be handled with some continuity. Any longitudinal design examining specific reform issues 
will have to be worked into the existing SASS structure and this presents some obstacles 
Operational costs of a longitudinal survey of districts and schools would probably not be significantly 
higher than those of the current design, but there could be some additional costs to a longitudinal design. 
Further, costs to develop items that maximize the time dimension of a longitudinal design on reform can be 
considerable. 

What has been clear so far from the analysis of the SASS overlap sample (see attached Feasibility Study 
Results), which just repeated the SASS cross-sectional items over a three year span, is that there is not much 
basic structural change in schools (i.e. grad-span, school size, staff size). Probably most of the short-term 
change in schools is in policies and practices. For example, NCES’s study of tracking policies found that 
schools within a five year period often changed their tracking practices (Carey, Farris, and Carpenter, 

1994); almost sixty percent of schools had changed their tracking policy in the last five years and about one 
half of schools were considering new modifications. 

Changes in school operation, not structure, are precisely where most implemented reforms would exist, but 
operations of schools are hard to measure. Also they may change often, particularly in politically contested 
areas of schooling. Since the current SASS does not focus very much on the dynamics of school policy 
setting and operation, there would probably not be much added information from just repeating current 
SASS items to a full panel of schools. To make SASS items yield more information, particularly about 
policy and practices, would require extensive redesign, especially for the proposed six-year spell between 
data collection waves. 

For example, although its not recommended here, one could propose to resample the entire 1993-94 
sample, adding retrospective items on policy and practice changes related to specific reform issues such as 
school-based management. Developing answerable, six-year retrospective items on school operation for all 
of the different types of schools included in the full SASS sample would be a large, and perhaps costly, 
undertaking. Items which require schools to dip into their institutional memory are easily prone to 
measurement error, therefore, valid items would take extra design efforts. Further there are response burden 
costs hidden in an extensive retrospective panel study. Items which are more sensitive to time issues 
usually require more response time, often significantly so. 

II. Are the costs worth the benefits? 

Would the costs for significant longitudinal redesign be worth it in terms of information payoff for NCES? 
Certainly school organizational reform is politically important to a large part of the American educational 
establishment and the public. And as described in Pechman et al (1996), many of the current reforms focus 
directly on the general areas of school management that SASS tried, in a limited way, to capture on the last 
two waves. Also as described above, longitudinal designs have some appeal to collecting information to 
answer key questions about the impact, the spread, and variation in reforms in districts and schools. 
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But at the same time, to justify the costs of a large, longitudinal redesign, NCES would have to be 
convinced that reports on the more complex nature of school change, as outlined above, are worth the 
investment. Keep in mind too, that in the past the agency has not taken full advantage of the longitudinal 
nature of its surveys of students. To maximize an investment in a longitudinal survey of schools on reform, 
NCES would have to plan to produce far more complex analyses and reports than it currently does. 

IV. Are there alternative designs? 

It may be better to consider some alternatives to a full SASS panel design. Instead of undertaking the costs 
to redesign the entire SASS to be an effective longitudinal survey on reform and other issues, perhaps 
NCES should consider smaller-scale, focused studies built upon the large cross-section school sample of 
SASS. While there would still be some design costs associated with developing useful items, a smaller, 
more focused approach to gathering school reform information over time might be more manageable. 

For example, in the curricular tracking FRSS, a smaller, nationally representative sample of public high 
schools (n=990) that had participated in the 1990-91 SASS collection was surveyed in 1993 as to their 
tracking policies and practices. The questionnaire included some explicitly longitudinal items on tracking, 
such as how often a school’s tracking policy had been modified in the last five years. And the survey could 
have included many more, such as specific histories of policy change. All of the other considerable 
information about each school from the SASS collection was available for analysis of the tracking issues. 
(The analysis of the overlap sample shows that most of the demographics and the basic organization of 
schools does not change over three years.) 

A similar design could be considered for reform issues. A small subset of schools from the full SASS 
sample could be resurveyed, perhaps annually, with a focus on reform issues. Almost all of the response 
time could be dedicated to the reform issues, thus keeping response burden relatively low. Also designing a 
special questionnaire, to be administered at a different time, to a smaller set of schools, will probably 
require less effort than trying to place longitudinal items within the main SASS data collection for all 
schools. Of course, this smaller sample would not support state level estimates. But here too NCES should 
consider some alternatives. Some selection of useful comparisons across a small number of states as to 
reform issues might serve as the basis of a small longitudinal add-on to SASS. 

V. If some alternative longitudinal design is used, what are the other 
design issues that need to be considered? 

1. Which SASS sample should an alternative study begin with? 

There are two alternatives. One could resample part of the 1993-94 SASS school sample and develop new 
reform items. This could be done as a focused FRSS-type survey before the 1999-2000 collection and then 
be followed-up as part of the main 1999-2000 collection and then again in the 2005 SASS. 

Selected schools would provide only current reform information instead of retrospective information back to 
1993-94, but the basic SASS structural information from 1993-94 could be used. Some baseline 
implementation information could be ready even before the 1999-2000 SASS, but any outcome 
information would still have to wait until after 1999-2000 collection. The advantage is that this provides 
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some information soon, but with the cost of adding a significant design task that is concurrent with the 
planning of the rest of the next SASS. The other option is to start any alternative studies with the 
1999-2000 collection. This means a longer wait for both implementation and outcome information, but 
provides more time to think through design and content issues. 

2. Which kinds of schools should a study include? 

Although this question is not necessarily related to whether or not some sort of longitudinal design should 
be used to collect information on reforms, it is worth considering here too. Most of the reform debates have 
been aimed at the public sector of K- 12 schooling. Although the private sector has played a role in the 
debate as points of comparisons, not much if anything has been said about whether current reforms with 
have an impact on private schools. Regardless of the prevailing myths, private schools may need reforms 
too. Also, as a recent SASS report indicates, large portions of the private sector are organizationally, and 
perhaps operationally, not much different from large portions of the private sector are organizationally, and 
perhaps operationally, not much different from large portions (Baker et al. 1996) of the public sector. Thus 
current reforms could be relevant to many private schools. Further, SASS has always reported on the full 
range of schools public and private, including all types of private schools and public schools such as schools 
for Native Americans. But the diversity of private schools might make it difficult to develop useable items 
on reforms. 



3. Broad versus focused reforms? 

Another design issue to be considered is which reforms? A focused smaller study lends itself to covering 
just a few key reform issues. 

4. National versus state? 

As mentioned above, it would not be possible in a smaller sample to have representative samples from each 
state. Rather certainly nationally representative samples are crucial, and if there are resources available, a 
selected number of states that yield interesting comparisons might be useful. 

Recomm endations 

Although some sort of longitudinal design for SASS would provide useful information on educational 
reform, both the hidden design costs and loss of continuity in the cross-sectional time series probably does 
not warrant a full panel, longitudinal study for the entire SASS. 

Instead, small-scale, longitudinal designs that are alternatives to a full panel design should be considered as 
a way to capture some important time varying information on reform. 

Operational costs for a panel design of districts and schools are not significantly higher than those for a 
repeated cross-section design, but costs to re-design SASS items to take advantage of the time dimension of 
a panel design can be substantial. Also truly longitudinal items may significantly increase response burden. 
It is probably not worth the effort to design retrospective items for any sample of 1993-94 schools, rather 
start prospectively, either with a sub-sample of the 1993-94 as baseline, or with a subsample of the 
1999-2000 as baseline. 
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Organizational impact (outcomes) of reforms will be relatively hard to measure in SASS, regardless of the 
longitudinal nature of the design. But information on impetuses and implementations of policy and practice 
associated with many current reforms can be accurately collected, and some longitudinal component would 
strengthen NCES reporting capabilities. 
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